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FORWORD 


The  FY  1991  Defense  Appropriation  Bill,  directed  DoD  to  develop  a  pilot 
program  to  evaluate  the  use  of  two  year  colleges  and  teletraining  to  deliver  military 
programs  of  instruction  to  members  of  the  armed  services  in  Florida.  The  program  was 
initiated  by  Dr.  Steven  Skiles  of  the  Defense  Training  and  Performance  Data  Center 
(TPDC),  and  completed  under  the  direction  of  Mr.  Bill  West,  after  the  program  was 
transferred  to  the  control  of  the  Defense  Institute  for  Training  Resources  Analysis 
(DITRA).  The  research  was  conducted  by  the  Institute  for  Simulation  and  Training  of  the 
University  of  Central  Florida. 


The  final  report  of  this  research  project  documents  the  training  performance 
achieved  and  costs  demonstrated  on  the  three  Army  and  two  Navy  courses  chosen  for 
presentation  in  the  program.  As  with  any  relatively  small  research  effort,  caution  must  be 
exercised  in  making  inferences  beyond  the  population  of  specific  courses  taught  and 
students  participating  in  the  study.  This  is  especially  true  with  respect  to  the  costs 
demonstrated  for  the  particular  courses  chosen.  The  costs  presented  within  the  report  are 
an  accurate  representation  of  what  was  expended  in  the  preparation  and  presentation  of 
these  classes  on  a  one  time  basis.  A  relative  cost  comparison  on  the  expected  life  cycle 
costs  of  video  teletraining  and  "standard”  forms  of  military  instruction  was  not 
accomplished,  nor  originally  intended  to  be  part  of  this  research  effort. 


This  research  provides  important  information  needed  by  the  DoD  in  its  ongoing 
investigations  on  providing  the  most  effective  and  cost  efficient  methods  of  maintaining 
the  training  readiness  of  the  Total  Force  of  the  DoD.  It  has  also  identified  again,  the  need 
for  high  quality  measures  of  training  effectiveness  to  use  in  comparing  alternative  methods 
of  training  presentation.  Without  these  measures,  cost  and  effectiveness  comparisons  will 
continue  to  be  done  on  a  subjective  bases. 


NTIS  CRA&I  M 

DTIC  TAB  □ 

Unannounced  □ 

Justification 


By . . . . 

Distnbution/ 


Availability  Codes 
Avail  c  '  o- 


O 


ACKNOWLEDGEMENTS 


This  report  was  made  possible  through  government  contract  No.  N61339-85-D-0024, 
Delivery  Order  10  awarded  by  the  Defense  Training  and  Performance  Data  Center  (TP DC)  and 
the  Defense  Institute  for  Training  Resources  Analysis  (DITRA)  to  the  Institute  for  Simulation  and 
Training  (1ST)  at  the  University  of  Central  Florida  (UCF).  Overall  project  direction  was  provided 
by  Dr.  Stephen  C.  Skiles  and  Mr.  Bill  West.  The  1ST  project  direction  was  provided  by  Dr. 
William  J.  Bramble  and  Dr.  Barbara  L.  Martin.  This  report  was  produced  by  1ST,  with  assistance 
from  the  Florida  Community  College  at  Jacksonville  (FCCJ). 

Special  thanks  are  due  to  Ms.  Marlene  Camper,  Project  Director  at  FCCJ,  Mr.  Doug 
Kerley,  Instructional  Manager  at  FCCJ,  and  Ms.  Kathy  Clower,  Media  Production  Manager  at 
FCCJ  for  their  input  into  this  report.  Their  insights  and  assistance  contributed  to  the  conclusions 
contained  within. 

Thanks  are  also  due  to  Ms.  Shannon  Duffy  and  Mr.  Eric  Llewellyn,  Student  Assistants, 
and  Ms.  Mary  Lou  Grover,  Project  Secretary  at  1ST,  for  their  patience  and  expertise  in  editing 
the  report  and  producing  the  tables,  figures,  and  final  version  of  this  report. 


iii 


EXECUTIVE  SUMMARY 


Introduction 

The  Department  of  Defense  (DoD)  was  directed  by  Congress  in  the  FY  1991  Department 
of  Defense  Appropriations  Bill  to  conduct  a  pilot  program  to  test  the  feasibility  of  using  a 
telecommunications  training  network  to  deliver  military  programs  of  instruction  to  the  armed 
services  in  Florida.  The  DoD  determined  that  this  program  would  employ  a  video  teletraining 
(VTT)  network  to  deliver  instruction  that  addressed  specific  military  duties  and  would  be 
presented  to  the  Active  Components  (AC)  and  Reserve  Components  (RC)  of  the  services. 

This  program  was  to  be  conducted  by  two-year  community  colleges  in  collaboration  with 
armed  forces  schools.  Local  colleges  and  military  units  are  a  natural  partnership  because  of  the 
need  for  military  units  to  retrain,  for  military  personnel  to  earn  college  credit,  and  because  of  the 
reservists’  limited  training  availability  (reservists  are  only  available  for  the  equivalent  of  thirty- 
eight  calendar  days  a  year  (Watt,  1988)).  Watt  stated  that  the  reserve  forces  are  an  underserved 
educational  population  and  that  local  educational  institutions  can  meet  many  of  the  reservists’ 
needs.  Evidence  exists  that  such  partnerships  can  work  and  that  they  can  provide  many  benefits 
to  both  the  college  and  the  reserve  unit.  The  Florida  Teletraining  Project  (FTP)  was  designed 
in  part  to  test  the  capability  of  the  community  colleges  to  provide  VTT  instruction  to  reservists. 


Objectives 

The  FTP  was  directed  to  identify,  collect,  and  evaluate  telecommunications  and  pilot 
project  test  data.  The  specific  project  objectives  were  to: 

•  ascertain  the  merit  of  using  telecommunications  training  provided  by  non-military 
sources  (i.e.,  community  colleges)  for  training  military  personnel 

•  quantify  the  value  of  the  instruction  received 

•  guide  future  government  and  DoD  decisions  related  to  distance  learning. 

Twenty-one  evaluation  objectives  were  identified  for  the  project  in  the  following 
categories:  technology,  instruction,  community  colleges,  and  cost.  Data  related  to  each  of  the 
objectives  was  collected  from:  (a)  students,  (b)  course  developers  who  were  also  the  VTT 
instructors,  (c)  all  remote  site  personnel  including  Instructional  Coordinators  (ICs),  Points  of 
Contact  (POCs),  and  technicians,  (d)  administrative  project  staff  at  the  origination  site,  (e) 
technical  and  production  teams,  and  (f)  military  personnel  including  subject  matter  experts 
(SMEs),  on-camera  military  instructors,  and  military  remote  site  personnel. 
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These  evaluation  objectives  focused  on  the: 


•  technology  selected  for  the  project  (e.g.,  ascertaining  its  reliability,  the  target 
community’s  acceptance  of  it,  and  how  appropriate  it  was  for  providing  the 
training) 

•  instruction  (e.g.,  determining  whether  or  not  the  students  met  the  learning 
objectives,  and  the  effectiveness  of  the  instructional  strategies) 

•  community  college  (e.g.,  roles  and  responsibilities  as  providers  of  military) 
instruction 

•  costs  (e.g.,  developing  a  cost  model,  determining  the  cost  of  course  design  and 
implementation,  estimating  the  cost  of  an  operational  system,  and  comparing  the 
cost  of  a  telecommunications  approach  to  the  cost  of  selected  conventional 
training  options). 

Thus,  the  purpose  of  the  FTP  was  to  assess  the  feasibility  of  using  two-year  community 
colleges  to  offer  military  courses  to  militaiy  personnel  using  a  telecommunications  network. 
Initially,  the  Training  and  Performance  Data  Center  (TPDC)  was  the  coordinating  government 
agency  for  the  project.  However,  when  TPDC  was  deactivated  approximately  one-third  of  the 
way  through  the  project,  the  Defense  Institute  for  Training  Resources  Analysis  (DITRA)  in 
Monterey,  California,  coordinated  the  remainder  of  the  project.  The  Institute  for  Simulation  and 
Training  (1ST)  in  Orlando,  Florida,  was  the  prime  contractor,  and  the  Florida  Community  College 
at  Jacksonville  (FCCJ)  was  the  primary  subcontractor. 

A  number  of  military  and  civilian  organizations  participated  in  the  FTP.  In  the  primary 
roles  of  design,  development,  and  implementation  of  the  instruction  were  the  community  colleges 
and  U.S.  Army  Reserve  Forces  (USARF)  schools,  three  Army  proponent  schools,  and  two  Chief, 
Naval  Education  and  Training  (CNET)  schools. 

Course  Reconfiguration 

Five  courses  were  reconfigured  for  delivery  on  the  U.S.  Army  Teletraining  Network 
(TNET).  Three  U.S.  Army  Reserve  Component  Configured  Courseware  (RC3)  Military 
Occupational  Specialty  (MOS)  courses  were  conducted:  71L10,  Administrative  Specialist; 
76Y10,  Unit  Supply  Specialist;  and  95B10,  Basic  Military  Police.  These  courses  were  delivered 
once  each  to  Army  National  Guard  and  Army  Reserve  soldiers  who  were  seeking  to  be 
reclassified  in  these  MOSs.  Two  Navy  special  topics  courses  were  conducted:  Handling 
Hazardous  Waste-Activity  Level  (HazWaste)  and  Total  Quality  Leadership  (TQL).  The  latter 
courses  addressed  joint  services  needs  and  were  made  available  to  members  of  interested  services 
and  components.  HazWaste  and  TQL  were  offered  three  and  two  times  respectively. 

The  courses  were  delivered  to  three  Florida  community  college  remote  sites:  St. 
Petersburg  Junior  College  (SPJC),  Valencia  Community  College  (VCC)  in  Orlando,  and  at  FCCJ. 


HazWaste  and  TQL  were  also  offered  at  two  out-of-state  sites  during  the  final  administrations 
of  these  courses:  Ft  Taylor  Hardin  in  Montgomery,  Alabama  and  Camp  Fogarty  in  East 
Greenwich,  Rhode  Island. 


The  five  courses  selected  for  VTT  delivery  had  to  be  converted  from  the  standard  mode 
of  platform  delivery  to  VTT  delivery.  The  five-component  Systems  Approach  to  Training  (SAT) 
model  was  adapted  for  use  in  reconfiguring  the  courseware.  In  the  adapted  model,  the  five 
functions  of  the  SAT  were  included  (i.e.,  Analysis,  Design,  Development,  Implementation,  and 
Evaluation);  due  to  the  complexity  of  the  reconfiguration  effort,  two  functions  were  added 
(Revise  Instruction  and  Management).  The  adapted  model  is  based  on  learning  and  instructional 
theory. 


The  five  courses  that  were  reconfigured  in  the  FTP  were  assigned  by  the  government. 
Therefore,  the  traditional  tasks  performed  during  the  analysis  phase  of  the  SAT  model  were  not 
conducted  during  this  project.  However,  these  courses  were  analyzed  for  their  suitability  for 
VTT.  During  the  design  phase,  the  Programs  of  Instruction  (POIs)  and  syllabi  were  analyzed  to 
determine  the  adequacy  of  the  course  materials  from  an  instructional  design  perspective.  In  the 
Development  Phase,  the  instruction  was  developed  and  produced.  All  course  materials  were 
developed  during  this  phase  of  reconfiguration.  Preparation  for,  and  the  delivery  of,  instruction 
were  the  primary  goals  of  the  implementation  phase  of  the  reconfiguration  process.  The  Revise 
Instruction  Phase  was  added  to  the  adapted  model  because  several  revision  and  validation  cycles 
had  to  be  performed  during  the  reconfiguration  process.  The  Management  Phase  was  also  added 
to  the  adapted  SAT  model  because  of  the  complexity  of  the  project.  Three  primary  groups  of 
people,  with  a  web  of  complex  relationships  among  them,  had  to  coordinate  for  the  success  of 
the  project.  These  groups  included:  (a)  die  design,  development,  production,  implementation, 
and  evaluation  teams,  (b)  the  military  organizations  and  groups  involved  in  the  project,  and  (c) 
the  three  community  colleges.  The  project  director  at  1ST  and  the  project  manager  at  DITRA 
were  responsible  for  these  tasks. 

A  considerable  amount  of  data  was  collected  during  delivery  of  the  courses  (Evaluation 
Phase).  One  of  the  roles  of  project  personnel  was  to  collect  data  from  students,  and  data  were 
collected  from  all  project  personnel.  The  results  of  these  evaluation  activities  are  the  primary 
focus  of  this  report. 


Course  Delivery 

Instruction  was  presented  live  over  TNET.  At  the  origination  site,  the  primary  person 
responsible  for  content  presentation  was  the  VTT  Instructor.  The  content  of  the  courses, 
however,  required  that  a  military  instructor/SME,  the  Military  Instructional  Assistant  (MIA), 
deliver  some  of  the  content.  During  delivery,  up  to  three  people  were  needed  to  implement  the 
instruction. 

At  each  remote  site,  a  community  college  site  coordinator,  known  as  the  Instructional 
Coordinator  (IC),  was  needed  as  the  instructor  of  record  for  each  course.  The  IC  also  performed 
some  of  the  off-line  instructional  roles.  For  the  MOS  courses,  a  Military  Site  Coordinator  (MSC) 
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was  also  required.  The  IC  and  MSC  were  the  VTT  Instructors’  representatives  at  the  remote 
sites.  It  was  their  responsibility  to  manage  the  instructional  activities  at  each  remote  site. 

Background 

Distance  Education  and  Video  Teletraining 

The  need  for  increased  training  has  prompted  the  military  services,  industry,  and  academia 
to  research  several  different  distance  education  strategies,  including  audio  teleconferencing, 
computer-based  teleconferencing,  and  VTT.  While  the  research  is  in  its  infancy  and  many  of  the 
studies  have  limitations,  researchers  have  drawn  two  primary  conclusions:  (a)  students  typically 
do  as  well  on  learning  outcomes  using  distance  education  methods  as  they  do  when  taught  by 
conventional  methods,  and  (b)  student  satisfaction  using  distance  education  is  equal  to  or  higher 
than  classroom  instruction  (DeLoughry,  1988;  Fahl,  1983;  Grimes,  Neilsen,  &  Niss,  1988;  Keene 
&  Cary,  1990;  Kruh,  1983;  Partin  &  Atkins,  1984).  The  need  to  find  cost  effective  and  efficient 
learning  strategies  has  led  to  conclusions  about  what  technologies  are  viable  and  effective,  what 
courses  should  be  selected  for  distance  education,  and  what  variables  are  key  when  designing 
distance  learning  courses. 

Communication  technologies  now  exist  to  present  instruction  via  distance  learning.  VTT, 
also  known  as  interactive  television,  is  an  application  of  distance  education  presented  by  a  two- 
way  audio  and  two-way  video  system.  Using  a  myriad  of  media-print,  audio,  video,  computers, 
supplemental  technologies  (e.g.,  the  telephone  and  facsimile  machine),  and  off-line  equipment 
(e.g.,  video  players)--provides  instructors  and  students  with  a  means  for  communicating  and 
learning  that  may  be  almost  as  good  as  being  there. 

The  major  advantage  of  VTT  is  that  it  offers  live  instruction.  While  other  technologies 
may  be  both  viable  and  cost  effective,  the  option  of  live  instruction  available  through  VTT  may 
outweigh  other  distance  learning  strategies.  However,  the  choice  of  technologies  may  also  be 
partly  dependent  on  the  types  of  courses  selected  for  distance  education.  Both  the  Navy  and  the 
Army  have  used  VTT  to  provide  faster,  more  efficient,  and  more  cost  effective  training. 

TRADOC  (1987)  supports  the  use  of  VTT  and  maintains  that  it  is  both  an  effective  and 
cost  efficient  method  for  presenting  instruction  to  RC  forces.  The  Naval  Air  Warfare  Center, 
Training  Systems  Division  (NAWCTSD),  formerly  the  Naval  Training  Systems  Center  (NTSC), 
has  also  conducted  a  series  of  studies  that  addressed  different  aspects  of  VTT.  They  too 
concluded  that  VTT  was  a  viable  alternative  for  providing  high  quality  and  cost  effective 
instruction  to  distance  learners. 

General  VTT  Recommendations 

The  NAWCTSD  developed  a  course  selection  model  for  Navy  VTT.  The  major 
recommendations  from  this  study  were:  (a)  select  courses  that  have  a  high  potential  for  savings, 
e.g.,  courses  with  high  throughput  and  short  duration,  (b)  select  courses  that  have  an  appropriate 


Vlll 


mix  of  lecture  and  laboratory,  (c)  do  not  select  courses  that  are  equipment  intensive,  and  (d)  do 
not  select  courses  that  require  substantial  curriculum  modification. 

Studies  conducted  by  the  Army  and  the  Navy  have  concluded  that  distance  education 
courses  typically  require  more  extensive  planning  than  platform  instruction.  Haarland  and  Newby 
(1984)  state  that  the  increases  in  student  performance  and  satisfaction  may  be  due  to  improved 
course  design  and  teaching  performance  rather  than  as  a  function  of  a  specific  technology. 
TRADOC  endorses  the  SAT  model  for  the  design  of  courses  and  states  that  effective  course 
delivery  must  take  into  account  proper  management  of  the  design,  resourcing,  development, 
production,  distribution,  and  evaluation  of  VTT  programs. 

At  the  core  of  interactive  television  is  the  concept  of  interactivity  (Moore,  1989;  Ritchie 
&  Newby,  1989;  Stoffel,  1987).  Interactive  television  is  defined  by  the  fact  that  good  instruction, 
whether  it  is  presented  in  a  classroom  or  at  a  distance,  stresses  interaction  among  the  participants 
in  the  teaching-learning  process. 

In  addition  to  interactivity,  the  following  course  design  features  are  also  necessary  for 
successful  VTT: 

•  group  dynamics  should  be  addressed 

•  student  involvement  activities  need  to  be  carefully  structured 

•  lecture  segments  should  not  exceed  20  minutes 

•  visual  aids  must  be  adapted  for  television  viewing 

•  careful  planning  is  required  to  handle  student  questions  and  discussions 

•  instructors  must  involve  and  motivate  learners 

•  an  interactive  study  guide  (ISG)  must  be  provided 

Because  military  projects  are  often  large,  management  issues  when  using  VTT  are  of 
utmost  importance.  Maloy  and  Perry  (1991)  addressed  the  policy  and  management  issues  of  a 
large  Navy  project.  They  found  that  a  large  teletraining  project  required  a  team  approach.  Some 
of  the  key  team  members  were  an  educational  specialist  (e.g.,  an  instructional  designer  or 
evaluation  expert),  an  engineer,  a  budget  analyst,  an  audio-visual  specialist,  a  security  specialist, 
a  researcher/analyst,  a  resource  sponsor  specialist  (to  provide  support  at  the  highest  level),  and 
representatives  from  civilian  personnel,  fleets,  training  command,  and  reserves. 

In  summary,  effective  teletraining  courses  are  similar  to  other  effective  distance  education 
programs.  Among  other  things,  they  must  be  well-designed,  provide  opportunities  for  interaction 
and  involvement,  make  good  use  of  visuals,  and  be  media-based. 


IX 


Evaluation  Methodology 


Students 

DITRA  coordinated  the  selection  of  the  students  for  this  project.  Students  were  selected 
by  their  respective  military  commands;  for  example,  the  8 1st  Army  Reserve  Command  (ARCOM) 
and  the  Florida  Army  National  Guard  (FLARNG).  A  total  of  275  students  were  trained  during 
the  project.  All  four  services  (including  the  reserve  components  of  the  Army  and  the  Air  Force), 
and  the  U.S.  Coast  Guard  were  involved:  FLARNG,  Florida  Air  National  Guard,  U.S.  Army 
Reserve,  U.S.  Navy  (military  and  civilian  personnel),  U.S.  Marine  Corps,  U.S.  Coast  Guard, 
Rhode  Island  Air  National  Guard,  and  Alabama  Army  National  Guard. 

The  average  age  of  all  the  students  was  33.37  years.  A  grade  of  E5  was  selected  as  an 
approximate  estimate  of  the  numbers  of  students  who  were  managers  versus  those  who  were  first 
line  supervisors  and  enlisted  personnel.  Approximately  63  percent  of  the  students  were  E5s  or 
below.  In  addition,  approximately  30  percent  of  the  students  had  a  duty  position  related  to  the 
course  in  which  they  were  enrolled  and  4.9%  had  a  civilian  occupation  related  to  the  course 
content.  All  of  the  students  were  high  school  graduates  or  the  equivalent  and  15%  had  a  four- 
year  college  degree  or  more. 

Students  were  asked  how  interested  they  were  in  the  course  content  prior  to  taking  the 
course.  Approximately  73  percent  rated  themselves  as  very  interested  (5  on  a  5-point  scale). 
When  asked  if  they  had  ever  taken  a  course  taught  by  television,  approximately  19  percent 
reported  that  they  had. 

Evaluation  Instruments 

There  were  40  different  data  gathering  instruments  developed  by  FTP  personnel  in 
conjunction  with  Systems  for  Training  and  Applied  Research,  Inc.,  Lexington,  Kentucky.  In 
addition,  six  standard  Army  forms  were  used  to  collect  the  test  data.  Performance  test  (PT) 
measures  were  used  in  evaluations  where  possible. 

Procedures 

A  complete  set  of  evaluation  forms  for  each  participant  (students  and  remote  site 
personnel)  in  each  of  the  MOS  courses  was  compiled  into  a  data  collection  notebook  and 
distributed  to  each  remote  site  on  the  first  day  of  each  course.  (Notebooks  were  not  needed  for 
the  special  topics  courses  because  all  the  data  was  collected  during  these  one  day  courses). 
Directions  were  provided  with  the  notebooks  that  specified  what  data  was  to  be  collected,  when 
it  was  to  be  collected,  and  from  whom. 

All  student  and  instructional  personnel  interviews  were  conducted  by  the  project 
evaluators.  Interviews  were  conducted  and  questionnaire  data  collected  according  to  a 
predetermined  schedule.  Individual  student  interviews  were  conducted  by  the  project  evaluators 
immediately  after  the  students  completed  the  questionnaire(s). 
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The  data  were  organized  by  categorizing  each  item  on  each  evaluation  form  according  to 
its  related  objective  or  objectives.  The  data  were  further  organized  by  determining  whether  each 
item  provided  data  that  was  course,  site,  or  student  related.  Broad  issues  related  to  each  objective 
were  also  identified.  This  resulted  in  a  data  matrix  for  each  evaluation  objective. 

The  evaluation  and  cost  data  were  coded  as  necessary  and  entered  into  a  database.  The 
Windows  version  of  the  Statistical  Package  for  Social  Sciences  (SPSS  for  Windows  5.0)  was 
used  to  analyze  the  evaluation  data. 


Conclusions 

As  previously  stated,  the  purpose  of  the  FTP  was  to  assess  the  feasibility  of  using  two- 
year  community  colleges  to  offer  military  programs  of  instruction  that  addressed  specific  military 
duties  and  content.  In  general,  the  TNET  system  was  reliable,  the  instruction  was  effective,  and 
the  community  colleges  were  able  to  present  high  quality  instruction.  In  this  project  VTT  was 
more  expensive  to  implement  than  resident  training.  The  costs  for  conducting  resident  training 
were  incomplete  (e.g.,  base  operating  costs  were  not  included).  Therefore,  the  comparisons  are 
skewed  in  favor  of  resident  training. 

Concerning  technology,  the  TNET  system  was  99%  reliable  and  possessed  high  quality. 
Students  and  ICs  indicated  that  they  preferred  the  VTT  approach  to  traditional  training. 
However,  even  with  training,  the  instructional  personnel  felt  that  more  training  and  practice  was 
needed  using  the  TNET  system. 

The  most  important  objective  was  the  quality  of  instruction.  All  students  passed  the 
learning  objectives,  and  over  90%  of  all  students  in  the  MOS  courses  passed  the  PTs  on  the  first 
attempt.  Students  rated  the  learning  methods  to  be  effective,  and  they  showed  interest  in,  and 
were  motivated  by  the  VTT  instruction.  Students  also  rated  the  instructional  personnel  as 
effective.  A  question  that  arose  was  whether  or  not  civilians,  with  their  lack  of  military  training, 
can  provide  quality  instruction  to  military  students.  The  answer  generally  appears  to  be  yes. 

Student  demographic  variables  did  not  predict  success  or  lack  of  it  for  military  students. 
Thus,  it  appears  that  VTT  is  an  acceptable  approach  for  the  general  military  population. 
However,  no  empirical  comparisons  could  be  made  between  the  performance  records  of  students 
in  the  VTT  courses  and  those  in  other  training  options. 

The  community  colleges  have  the  technical  and  instructional  capabilities  to  implement 
VTT  instruction.  The  technical  staff,  TV  studio,  and  production  facilities  were  excellent.  While 
the  faculty  at  FCCJ  (the  origination  site)  lacked  instructional  design  and  military  training 
expertise,  they  were  able  to  present  high  quality  instruction  given  specific  training. 

The  community  colleges  were  also  able  to  grant  academic  and  continuing  education  credit 
(CEC)  for  all  courses.  Although  not  all  students  were  interested  in  receiving  credit,  those  that 
were  indicated  that  college  credit  was  a  definite  advantage  to  having  military  training  provided 
at  a  community  college. 
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In  general,  during  this  project,  VTT  was  more  expensive  to  conduct  than  resident  training 
partly  due  to  the  expense  of  the  technology.  With  the  exception  of  the  HazWaste  course,  the 
VTT  courses  were  more  expensive  than  resident  training.  The  HazWaste  course  was  less 
expensive  because  there  were  a  large  number  of  students  in  the  course  and  because  of  travel  and 
per  diem  savings.  The  VTT  courses  become  more  cost  effective  if  the  same  course  is  presented 
more  than  once  because  design  and  development  costs  are  paid  only  once.  In  addition,  if  different 
courses  are  presented  in  the  same  month,  each  course  can  share  the  TNET  costs  for  that  month. 
However,  cost  alone  is  not  the  only  factor  to  consider  when  comparing  VTT  to  other  training 
options.  If  VTT  provides  students  greater  access  to  courses  they  might  not  otherwise  receive, 
and  if  college  credit  is  deemed  a  benefit  of  military  training,  then  higher  costs  for  VTT  may  be 
worth  the  additional  expenditure. 
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THE  FLORIDA  TELETRAINING  PROJECT:  FINAL  REPORT 
RESULTS  OF  THE  PILOT  TEST 


INTRODUCTION 
Scope  of  Work 

The  Department  of  Defense  (DoD)  was  directed  by  Congress  in  the  FY  1991  Department 
of  Defense  Appropriations  Bill  to  conduct  a  pilot  program  to  test  the  feasibility  of  using  a 
telecommunications  training  network  to  deliver  military  programs  of  instruction  to  the  armed 
services  in  Florida.  The  bill  stated  that  the  pilot  program  could  be  developed  and  implemented 
by  two-year  colleges  in  collaboration  with  armed  forces  schools. 

Having  been  tasked  to  carry  out  the  project,  the  DoD  determined  that  the  pilot  program 
would  satisfy  the  congressional  direction  by  using  a  teletraining  network  to  deliver  military 
programs  of  instruction:  (a)  to  both  the  Active  Components  (AC)  and  Reserve  Components  (RC) 
of  the  services  and  (b)  that  addressed  specific  military  duties.  In  addition,  it  was  directed  that 
the  program  should  have  a  scientifically  valid  evaluation.  The  project  was  directed  to  identify, 
collect,  and  evaluate  telecommunications  and  pilot  project  test  data  that  could  be  used  to: 

(1)  ascertain  the  merit  of  using  telecommunications  training  provided  by  non-military 
sources  (i.e.,  community  colleges)  for  training  military  personnel, 

(2)  quantify  the  value  of  the  instruction  received,  and 

(3)  guide  future  government  and  DoD  decisions  related  to  distance  learning. 

Purpose  and  Scope  of  the  Florida  Teletraining  Project 

The  purpose  of  the  Florida  Teletraining  Project  (FTP)  was  to  assess  the  feasibility  of 
using  two-year  community  colleges  to  offer  military  courses  to  military  personnel  using  a 
telecommunications  network.  The  instruction  was  delivered  from  the  Florida  Community  College 
at  Jacksonville  (FCCJ)  to  three  remote  sites:  Valencia  Community  College  in  Orlando  (VCC), 
St.  Petersburg  Junior  College  (SPJC),  and  a  remote  classroom  site  at  FCCJ.  The  Training 
Performance  and  Data  Center  (TPDC),  in  Orlando,  and  later,  the  Defense  Institute  for  Training 
Resources  Analysis  (DITRA)  in  Monterey,  California,  was  the  coordinating  government  agency 
for  the  project.  The  Institute  for  Simulation  and  Training  (1ST)  in  Orlando  was  the  prime 
contractor  for  the  project. 

Five  courses  were  reconfigured  for  delivery  on  the  U.S.  Army  Teletraining  Network 
(TNET).  Three  U.S.  Army  Reserve  Component  Configured  Courseware  (RC3)  Military 
Occupational  Specialty  (MOS)  courses  were:  71L10,  Administrative  Specialist;  76Y10,  Unit 
Supply  Specialist;  and  9SB10,  Basic  Military  Police.  These  courses  were  delivered  once  each  to 
Army  National  Guard  and  Army  Reserve  soldiers  who  were  seeking  to  be  reclassified  in  these 
MOSs.  Two  U.S.  Navy  special  topics  courses  were:  Handling  Hazardous  Waste-Activity  Level 
(HazWaste)  and  Total  Quality  Leadership  (TQL).  The  latter  courses  addressed  joint  services 
needs  and  were  made  available  to  members  of  interested  services  and  components,  i.e..  Army 
National  Guard,  U.S.  Marines,  Air  National  Guard,  U.S.  Navy,  and  U.S.  Coast  Guard.  HazWaste 
and  TQL  were  offered  three  and  two  times  respectively. 


Two  out-of-state  sites  participated  in  the  project  during  the  final  administration  of  the 
special  topics  courses:  Camp  Fogerty  (FOG)  in  Rhode  Island  and  Ft  Taylor  Hardin  (FTH)  in 
Alabama.  Both  HazWaste  and  TQL  were  presented  in  Rhode  Island.  Only  HazWaste  was 
presented  in  Alabama.  In  Rhode  Island,  the  students  who  participated  in  the  project  were  from 
four  different  branches  of  the  military:  Rhode  Island  Army  National  Guard,  U.S.  Marines,  Rhode 
Island  Air  National  Guard  and  the  U.S.  Navy.  In  Alabama,  the  students  enrolled  in  the  course 
were  Alabama  Army  National  Guard  and  U.S.  Marines. 

Purpose  of  this  Report 

The  purpose  of  this  report  is  to  present  the  evaluation  results  of  the  FTP.  A  separate 
document,  Florida  Teletraining  Project:  Reconfiguration  of  Military  Courses  for  Video 
Teletraining  Delivery  (Martin,  1993),  describes  the  processes  and  personnel  that  were  used  to 
design,  develop,  and  deliver  the  five  video  teletraining  (VTT)  courses.  This  report  is  available 
from  DITRA. 


Project  Participants 

A  number  of  military  and  civilian  organizations  participated  in  the  FTP.  In  the  primary 
roles  of  design,  development,  and  implementation  of  the  instruction  were  the  community  colleges 
and  U.S.  Army  Reserve  Forces  (USARF)  schools,  three  Army  proponent  schools,  and  two  Chief, 
Naval  Education  and  Training  (CNET)  schools.  These  participants,  plus  other  key  organizations 
are  listed  in  Table  1. 

Some  of  the  tasks  accomplished  by  various  military  organizations  were  to: 

•  identify  the  students 

•  arrange  for  student  participation 

•  select  appropriate  subject  matter  experts  (SMEs)  and  military  instructional 
assistants  (MIAs) 

•  provide  needed  course  and  reference  materials 

•  provide  military  content  and  doctrine  review 

•  define  and  establish  the  approval  processes  for  the  reconfigured  courses, 
and 

•  certify  the  students  and  the  instruction. 

The  primary  roles  of  the  civilian  organizations  were: 

•  1ST  was  the  prime  contractor  and  had  overall  responsibility  for  executing 
the  project.  1ST  also  provided  technical  assistance  for  the  project. 
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•  FCCJ  was  responsible  for  the  design,  development,  and  implementation  of 
the  courseware,  for  providing  one  remote  site  and,  for  insuring  that 
organizational  decisions  regarding  the  community  colleges  were  made. 

•  VCC  and  SPJC  each  provided  a  remote  site  for  receiving  the  instruction. 


TABLE  1 


PROJECT  PARTICIPANTS 


POD 

OASD  (FM&P),  DITRA 
National  Guard  Bureau 
U.S.  Army  Forces  Command 
U.S.  Army  Reserve  Command 
2nd  Army 

81st  Army  Reserve  Command 

Florida  National  Guard 

U.S.  Army  Training  and  Doctrine  Command 

U.S.  Army  Training  Support  Center 

U.S.  Army  Extension  Training  Directorate 

U.S.  Army  Military  Police  School 

U.S.  Army  Quartermaster  School 

U.S.  Army  Soldier  Support  Center 

Chief,  Naval  Education  and  Training 

Naval  Training  Systems  Center 

Commander,  Naval  Aviation  Activities,  Jacksonville 

Orlando  Naval  Training  Center 

Naval  Air  Station,  Cecil  Field,  Jacksonville 

Headquarters,  U.S.  Air  Force 

Marine  Corps  Combat  Development  Center 


CIVILIAN  EDUCATION  INSTITUTIONS 

University  of  Central  Florida,  Institute  for  Simulation  and  Training 
Florida  Community  College  at  Jacksonville 
Valencia  Community  College 
St.  Petersburg  Community  College 
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General  Project  Objectives 


The  general  project  objectives  were  to: 

•  ascertain  the  merit  of  using  telecommunications  training  provided  by  non-military 
personnel 

•  quantify  the  value  of  the  instruction  received 

•  provide  generic  lessons  learned  from  pilot  test 

•  provide  input  to  guide  further  government  and  DoD  decisions  related  to  distance 
learning. 

Twenty-one  separate  evaluation  objectives  were  identified  related  to  the  general  objectives 
including: 

•  Four  objectives  related  to  the  technology  selected  for  the  project 

•  Six  objectives  related  to  the  instruction 

•  Seven  objectives  related  to  community  college  roles  and  responsibilities  as 
providers  of  military  instruction 

•  Four  objectives  related  to  costs. 


Project  Management  and  Timeline 

The  Defense  Training  Performance  and  Data  Center  (TPDC)  and  DITRA  administered  the 
project.  The  timeline  shown  in  Figure  i  gives  the  key  events  of  the  FTP.  A  kickoff  conference 
hosted  by  TPDC  and  1ST  was  held  in  Orlando  in  November  1991  for  all  project  participants. 
In  December,  1991  the  course  design  and  development  process  began.  Design  and  development 
of  some  of  the  courses  occurred  simultaneously  with  course  delivery.  The  Army  MOS  courses 
were  presented  from  5  October  through  18  December  1992.  The  Navy  courses  were  presented 
from  27  January  through  25  February  1993.  Data  was  collected  from  3  August  1992  through 
October  1993.  Project  data  was  analyzed  and  the  final  report  written  from  February  1993  to 
January  1994. 
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Figure  1.  Project  Timeline:  Key  Milestones 


Organization  of  the  Report 

This  report  is  organized  into  six  major  sections:  (a)  background  and  overview  of  the 
project,  courses,  and  methodology,  (b)  results  of  the  data  related  to  the  technology,  (c)  results  of 
the  data  related  to  the  instruction,  (d)  results  of  the  data  related  to  the  community  colleges,  (e) 
results  of  the  cost  analysis,  and  (f)  summary  and  conclusions. 


BACKGROUND 


Teletraining  Research  in  the  Military 

Recent  interest  has  been  shown  by  the  military  in  methods  of  distributed  training,  i.e., 
courses  of  instruction  packaged  for  delivery  at  remote  locations,  including  VTT.  The  Training 
and  Doctrine  Command  (TRADOC)  (1990)  stated  that  the  goals  of  a  distributed  training  strategy 
were  to  "increase  training  opportunities,  improve  the  quality  of  instruction,  increase 
standardization,  and  reduce  the  time  soldiers  spend  away  from  their  units"  (p.  2-16). 

Two  of  the  key  reasons  for  reliance  on  a  distributed  training  strategy  are:  (a)  the 
importance  and  size  of  the  RC  and  (b)  dwindling  resources.  It  is  estimated  that  the  RC 
constitutes  more  than  half  of  the  Army’s  combat  arms  units  and  more  than  two-thirds  of  the 
Army’s  combat  support  and  combat  service  support  units  (TRADOC,  1990).  These  soldiers  must 
be  trained  to  AC  standards  yet  they  do  not  train  on  a  daily  basis.  Training  must  be  developed 
to  meet  both  the  Army’s  needs  and  the  time  frames  available  to  train  RC  soldiers. 

The  need  for  increased  training  has  prompted  the  military  services,  industry,  and  academia 
to  research  several  different  distance  education  strategies,  including  audio  teleconferencing, 
computer-based  teleconferencing,  and  video  teleconferencing.  While  the  research  is  in  its  infancy 
and  many  of  the  studies  have  limitations,  researchers  have  drawn  two  primary  conclusions:  (a) 
students  typically  do  as  well  on  learning  outcomes  using  distance  education  methods  as  they  do 
when  taught  by  conventional  methods,  and  (b)  student  satisfaction  using  distance  education  is 
equal  to  or  higher  than  classroom  instruction  (DeLoughry,  1988;  Fahl,  1983;  Grimes,  Neilsen, 
&  Niss,  1988;  Keene  &  Cary,  1990;  Kruh,  1983;  Partin  &  Atkins,  1984). 

The  need  to  find  cost  effective  and  efficient  learning  strategies  has  led  to  conclusions 
about  what  technologies  are  viable  and  effective,  what  courses  should  be  selected  for  VTT,  and 
what  variables  are  key  when  designing  distance  learning  courses. 

What  Technologies  are  Viable  and  Cost-effective? 

Communication  technologies  now  exist  to  present  instruction  via  distance  learning.  VTT, 
also  known  as  two-way  interactive  television,  is  an  application  of  distance  education  presented 
by  a  two-way  audio  and  two-way  video  system.  Using  a  myriad  of  media— print,  audio,  video, 
computers,  supplemental  technologies  (e.g.,  the  telephone  and  facsimile  machine),  and  off-line 
equipment  (e.g.,  video  players)— provides  instructors  and  students  with  a  means  for 
communicating  and  learning  that  may  be  almost  as  good  as  being  there. 

Both  the  Navy  and  the  Army  have  used  VTT  to  provide  faster,  more  efficient,  and  more 
cost  effective  training.  TRADOC  (1987)  supports  the  use  of  VTT  and  maintains  that  it  is  both 
an  effective  and  cost  efficient  method  for  presenting  instruction  to  RC  forces.  The  Naval  Air 
Warfare  Center,  Training  Systems  Division  (NAWCTSD),  formerly  the  Naval  Training  Systems 
Center  (NTSC)  has  also  conducted  a  series  of  studies  that  addressed  different  aspects  of  VTT 
(Bailey,  Sheppe,  Hodak,  Kruger  &  Smith,  1989;  McDonald,  Weisenford,  Fleeton,  Kreiner  & 
Hodak,  1990;  Sheppard,  Gonos,  Weisenford  &  Hodak,  1990).  They  too  concluded  that  VTT  was 
a  viable  alternative  for  providing  high  quality  and  cost  effective  instruction  to  distance  learners. 
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The  Army  has  two  VTT  systems,  the  TRADOC  Satellite  Education  Program  at  the  U.S. 
Army  Logistics  Management  Center  (ALMC)  at  Ft  Lee,  Virginia  and  the  TNET  system  operated 
by  the  Army  Extension  Training  Directorate  (AETD)  at  Ft  Eustis,  Virginia.  The  systems  are 
different  in  that  the  former  uses  one-way,  full  motion  analog  video  and  the  latter  uses  two-way, 
compressed  digitized  video.  Training  Strategies  for  the  90‘s  (TRADOC,  1991)  summarized  the 
studies  that  have  been  done  on  the  two  systems  and  stated  that  both  are  cost  and  learning 
effective.  The  report  stated,  "VTT  appears  to  be  the  solution  to  a  myriad  of  problems  involved 
in  sustaining  a  well-trained  force." 

The  U.S.  Navy  CNET  Electronic  Schoolhouse  Network  (CESN)  at  the  Fleet  Combat 
Training  Center,  Atlantic  (FCTCLANT)  located  in  Dam  Neck,  Virginia,  also  uses  compressed 
digitized  video,  however,  it  is  only  one  of  the  VTT  configurations  the  Navy  has  experimented 
with  (Maloy  &  Perry,  1991).  CESN  interconnects  major  fleet  training  facilities  (e.g., 
schoolhouses)  rather  than  distributing  training  over  a  widely  dispersed  network. 

In  addition  to  VTT,  the  military  services  have  developed  and  used  computer-based 
technologies  for  distance  education.  SMART,  System  for  Managing  Asynchronous  Remote 
Training,  was  one  of  the  successful  systems  developed  to  provide  computer-mediated  instruction 
to  reservists  (Harbour,  Daveline,  Wells,  Schurman,  &  Hahn,  1990).  The  advantage  of  VTT  is 
that  it  offers  live  instruction.  While  other  technologies  may  be  both  viable  and  cost  effective, 
the  option  of  live  instruction  available  through  VTT  may  outweigh  other  distance  learning 
strategies.  However,  the  choice  of  technologies  may  also  be  partly  dependent  on  the  types  of 
course  selected  for  distance  education. 

What  Courses  Should  be  Selected  for  Video  Teletraining? 

McDonald,  et  al.  (1990)  developed  a  course  selection  model  for  Navy  VTT  .  The  major 
recommendations  from  this  study  were:  (a)  select  courses  that  have  a  high  potential  for  savings, 
e.g.,  courses  with  high  throughput  and  short  duration,  (b)  select  courses  that  have  an  appropriate 
mix  of  lecture  and  laboratory,  (c)  do  not  select  courses  that  are  equipment  intensive,  and  (d)  do 
not  select  courses  that  require  substantial  curriculum  modification. 

One  reason  the  Navy’s  CESN  two-way  audio  and  video  teletraining  network  in  Dam 
Neck,  Virginia  has  been  found  to  be  both  cost  effective  and  efficient  is  because  of  appropriate 
course  selection  (CNET  Handout,  1992).  System  utilization  increased  from  46%  in  1989  to  90% 
in  1992.  The  factors  that  raised  the  utilization  rate  were  delivery  of  teletraining  courses  that: 

*  were  single  sited  (delivered  by  one  single  schoolhouse  and  only  presented  via 
CESN) 

•  had  high  demand  (throughput) 

•  averaged  about  one  week  in  length 

*  were  designed  to  address  cognitive  rather  than  psychomotor  skills. 


What  are  the  Key  Design  Variables  for  Distance  Learning 
and  Video  Teletraining  Courses? 

Studies  conducted  by  the  Army  and  the  Navy  conclude  that  distance  education  courses 
typically  require  more  extensive  planning  than  platform  instruction.  Haarland  and  Newby  (1984) 
stated  that  the  increases  in  student  performance  and  satisfaction  found  in  the  research  may  be  due 
to  improved  course  design  and  teaching  performance  rather  than  as  a  function  of  a  specific 
technology.  TRADOC  endorses  the  systems  approach  to  training  (SAT)  for  the  design  of  courses 
and  states  that  effective  course  delivery  must  take  into  account  proper  management  of  the  design, 
resourcing,  development,  production,  distribution,  and  evaluation  of  VTT  programs. 

In  a  comprehensive  review  of  the  literature  on  computer-mediated  communications 
(CMC),  Wells  (1990)  discussed  the  factors  relevant  for  an  instructor  and  course  designer  in  a 
distance  learning  environment.  She  stated  that:  (a)  students  and  instructors  must  be  provided  with 
an  orientation  to  CMC  in  order  for  a  program  to  be  effective,  (b)  instructors  must  play  a 
motivational  and  facilitative  role,  including  maintaining  personal  contact  with  students,  (c) 
students  must  be  provided  with  consistent  and  regular  feedback,  and  (d)  interaction  with  other 
students  typically  has  a  positive  effect  on  completion  rates  and  performance.  In  addition,  she 
lists  other  important  factors  for  successful  distance  education:  (e)  topics  that  lend  themselves  to 
discussion  rather  than  rote  memorization  are  best,  (f)  the  first  unit  of  a  course  should  not  be  too 
long  or  more  difficult  than  later  units,  (g)  using  a  variety  of  media  is  important,  (h)  completion 
rates  are  generally  higher  when  the  instructor  rather  than  the  student  sets  the  schedule  and  pace, 
and  (i)  a  successful  CMC  class  is  not  dependent  upon  any  face-to-face  meetings. 

At  the  core  of  VTT  is  the  concept  of  interactivity.  Good  instruction,  whether  it  is 
presented  in  a  classroom  or  at  a  distance,  stresses  interaction  among  the  participants  in  the 
teaching-learning  process.  Moore  (1989)  stated  that  there  are  three  primary  types  of  interaction: 
student  to  teacher,  teacher  to  student,  and  student  to  student.  Each  type  should  be  planned  for 
during  instruction.  Stoffel  (1987)  also  stated  that  student  interaction  with  the  instructional 
materials  leads  to  positive  outcomes  in  satisfaction  and  achievement.  Some  investigations  have 
shown  that  in  classrooms  with  higher  levels  of  interaction,  students  have  higher  levels  of 
achievement  and  more  positive  attitudes  (Ritchie  &  Newby,  1989). 

In  addition  to  interactivity,  the  following  course  design  features  are  also  necessary  for 
successful  VTT  (Bailey,  et  al.,  1989;  Cyrs  and  Smith,  1990;  Defense  Language  Institute,  1992; 
McDonald,  et  al.,  1990;  Ostendorf,  1991;  Sheppard,  et  al.,  1990; ): 

•  group  dynamics  should  be  addressed 

•  student  involvement  activities  need  to  be  carefully  structured 

•  lecture  segments  should  not  exceed  20  minutes 

•  visual  aids  must  be  adapted  for  television  viewing 

•  careful  planning  is  required  to  handle  student  questions  and  discussions 


•  instructors  must  involve  and  motivate  learners  and 

•  an  interactive  study  guide  (ISG)  must  be  provided. 

Because  military  projects  are  often  large,  management  issues  are  of  utmost  importance 
when  using  VTT.  Maloy  and  Perry  (1991)  addressed  the  policy  and  management  issues  of  a 
large  Navy  project.  They  found  that  a  large  teletraining  project  required  a  team  approach.  Some 
of  the  key  team  members  were  an  educational  specialist  (e.g.,  an  instructional  designer  or 
evaluation  expert),  an  engineer,  a  budget  analyst,  an  audio-visual  specialist,  a  security  specialist, 
a  researcher/analyst,  a  resource  sponsor  specialist  (to  provide  support  at  the  highest  level),  and 
representatives  from  civilian  personnel,  fleets,  training  commands,  and  reserves. 

In  summary,  effective  teletraining  courses  are  similar  to  other  effective  distance  education 
programs.  Among  other  things,  they  must  be  well-designed,  provide  opportunities  for  interaction 
and  involvement,  make  good  use  of  visuals,  and  be  media  based.  A  comprehensive  discussion 
of  course  design  issues  related  to  VTT  can  be  found  in  the  Reconfiguration  document  available 
from  DITRA. 

Summary  of  General  Issues  in  Video  Teletraining 

•  Select  courses  that  are  cognitive  rather  than  psychomotor  and  that  have  high 
demand. 

•  Course  design  and  development  is  a  critical  component  of  a  successful  VTT 
course.  Typically  VTT  courses  require  more  detailed  pre-planning  than  traditional 
courses.  Time  and  resources  must  be  allocated  to  the  pre-planning  stages. 

•  Courses  must  be  designed  to  facilitate  interaction,  to  provide  feedback,  to  be 
motivating,  and  to  humanize  the  instruction.  Course  developers  must  be 
knowledgeable  in  the  principles  of  learning  and  instruction. 

•  Instructional  personnel  are  one  of  the  most  important  aspects  of  a  VTT  course. 
They  may  require  considerable  training  and  practice  to  be  effective. 

•  It  takes  a  team  of  people  to  design  and  deliver  good  VTT  instruction.  This  team 
must  be  organized  and  the  roles  and  responsibilities  of  all  personnel  must  be 
delineated. 

•  Equipment  failures  occur.  Contingency  plans  must  be  prepared.  These  plans  may 
require  the  expenditure  of  additional  time  and  resources. 

•  In  large  scale  projects,  all  the  various  management  and  organization  functions 
need  to  be  coordinated.  Someone  who  is  knowledgeable  in  all  aspects  of  VTT 
needs  to  maintain  a  big  picture  perspective. 
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OVERVIEW  OF  THE  FLORIDA  TELETRAINING  PROJECT  COMPONENTS 


Description  of  the  Courses 

The  71L10  Unit  Administrative  Specialist  course  is  a  single-phase  course  that  can  be 
taught  in  either  an  Inactive  Duty  Training  (IDT)  Phase  or  an  Active  Duty  for  Training  (ADT) 
Phase  (U.S.  Army  Soldier  Support  Center,  1991).  This  course  was  presented  in  an  ADT  mode. 
It  was  a  73-hour  course  and  was  presented  during  a  two- week  block  from  17  October  to  31 
October  1992.  The  Administrative  Specialist  is  responsible  for  the  routine  office  administration 
of  an  activity.  He  or  she  works  at  various  organizational  levels  throughout  the  Army,  from 
company  through  division,  installation,  or  higher  headquarters. 

The  76Y10  Unit  Supply  Specialist  course  is  a  dual-phased  (IDT  and  ADT)  course  (U.S. 
Army  Quartermaster  Center  and  School,  1989).  Only  the  IDT  phase  was  presented  during  the 
project.  This  phase  of  the  course  was  96-hours  and  was  presented  during  a  two-week  block  from 
7  November  to  22  November  1992.  To  receive  the  MOS,  the  76Y10  student  must  also  take  the 
ADT  phase.  Arrangements  for  completing  the  ADT  phase  were  made  independently  of  this 
project.  The  Unit  Supply  Specialist  performs  unit  and  organization  supply  procedures.  These 
include  the  tasks  of  request;  receipt;  storage;  and  issue  and  accountability  of  expendable  and 
durable  supplies  and  equipment  (e.g.,  individual,  organizational,  installation). 

The  95B10  Basic  Military  Police  course  is  also  a  dual-phase  (IDT  and  ADT)  course 
(U.S.  Army  Military  Police  School,  1991).  Only  the  IDT  phase  was  taught  during  this  project. 
This  phase  of  the  course  was  66-hours  and  was  presented  during  a  two-week  block  from  5 
December  to  18  December  1992.  The  ADT  phase  was  presented  by  the  3391st  USARF  School 
at  Camp  Blanding,  Florida  in  the  summer  of  1993.  The  95B 10  Military  Police  (MPs)  are  soldiers 
who  perform  the  duties  of  entry  level  military  police.  The  entry  level  MP  performs  the  tasks  of 
apprehension  and  search,  patrol  and  traffic  operations,  investigations,  physical  security,  and  self- 
defense.  In  addition,  the  MP  prepares  and  gathers  military  police  information,  reports,  and  forms. 

Handling  Hazardous  Waste— Activity  Level  is  a  U.S.  Navy  course  specially  adapted  for 
the  FTP  from  the  Hazardous  Waste  Coordinator  course.  The  FTP  one-day  course  was  designed 
to  give  hazardous  waste  handlers  the  information  necessary  to  make  environmentally  and 
personally  safe  decisions  regarding  the  disposal  of  hazardous  and  regulated  wastes.  This  course 
was  offered  a  total  of  three  different  times,  twice  to  the  FTP  teletraining  remote  sites  on  27 
January  1993  and  5  February  1993,  and  one  additional  time  on  25  February  1993  to  the  three 
FTP  sites,  Camp  Fogerty  and  Ft  Taylor  Hardin.  The  HazWaste  course  topics  included  a  review 
of  pertinent  laws  and  regulations  and  a  discussion  of  the  physical  and  chemical  properties  of 
hazardous  materials,  the  correct  techniques  for  delivering  and  transferring  hazardous  materials 
at  the  hazardous  waste  collection  site,  and  pollution  and  spill  prevention. 

Total  Quality  Leadership  (TQL)  is  the  U.S.  Navy’s  adaptation  of  Dr.  W.  Edwards 
Deming’s  approach  to  continuous  quality  improvement  (Mr.  Jim  Miller,  Total  Quality  Leadership 
Curriculum  Developer,  CNET,  personal  communication,  June  2,  1993).  A  one-day  course  was 
presented  to  provide  an  introduction  to  and  an  awareness  of  the  U.S.  Navy’s  TQL  philosophy. 
This  course  was  offered  twice,  once  to  the  FTP  sites  on  30  January  1993,  and  once  on  22 
February  1993  to  the  three  FTP  sites  and  to  Camp  Fogerty.  The  topics  covered  during  the  course 
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included  the  background  of  TQL,  how  it  is  defined  by  the  Navy,  basic  principles,  methods  and 
tools  used  in  TQL,  and  how  the  Navy  has  implemented  it. 


Participating  Sites 

The  five  sites  participating  in  the  FTP  are  shown  in  Figure  2. 


Figure  2.  Florida  Teletraining  Project  Origination  and  Remote  Sites 
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FCCJ1  was  the  origination  site  for  all  five  courses  and  the  courses  were  presented  at  the 
following  sites: 


FCCJ1  FCCJ2  SPJC  VCC  FOG  FTH 


71L10  V  V  V  V 

76Y10  V  V  V  V 

95B10  V  V  V  V 

HAZ  V  V  V  V  V  V 

TQL  V  V  V  V  V 


Both  FCCI1  and  FCCJ2  were  located  in  the  Downtown  Campus  of  the  Florida 
Community  College  at  Jacksonville.  The  Downtown  Campus  has  a  television  studio  that  served 
as  the  origination  site  for  the  project.  The  remote  site,  FCCJ2,  was  located  in  an  upstairs 
classroom  in  the  same  building  as  the  origination  site.  Students  at  the  remote  site  did  not  know 
that  the  instruction  was  being  broadcast  from  the  same  building.  This  classroom  was  adapted 
for  use  as  a  pilot  test  site  and  official  visitation  center  for  the  project. 

The  remote  site  at  SPJC  was  located  at  the  Allstate  Center  which  houses  the  college’s 
Criminal  Justice  Institute.  The  VCC  remote  site  was  located  at  the  McCoy  Center  for  Business 
and  Industry  Services  which  is  adjacent  to  the  Orlando  Naval  Training  Center  Annex.  The 
remote  site  at  Ft  Taylor  Hardin  (FTH)  was  located  in  an  Alabama  Army  National  Guard  Armory 
in  Montgomery.  The  remote  site  in  Rhode  Island  was  located  at  Camp  Fogerty  National  Guard 
Training  Site  (FOG).  There  were  no  trained  technicians  at  the  out  of  state  sites. 


EVALUATION  METHODOLOGY 
Student  Population 

DITRA  coordinated  the  selection  of  students  for  this  research.  Students  were  selected  by 
their  respective  military  commands,  for  example,  the  81st  Army  Reserve  Command  (ARCOM) 
and  the  Florida  Army  National  Guard.  Table  2  shows  the  number  of  students,  by  military  service 
and  course,  who  were  enrolled  at  each  site  for  the  following  courses:  (a)  MOS,  (b)  HazWaste, 
and  (c)  TQL.  A  total  of  275  students  were  trained  during  the  project.  All  four  services 
(including  the  reserve  components  of  the  Army  and  the  Air  Force)  and  the  U.S.  Coast  Guard 
were  involved:  Florida  Army  National  Guard,  Florida  Air  National  Guard,  U.S.  Army  Reserve, 
U.S.  Navy  (military  and  civilian  personnel),  U.S.  Marine  Corps,  U.S.  Coast  Guard,  Rhode  Island 
Air  National  Guard,  and  Alabama  Army  National  Guard. 

Table  3  gives  a  composite  of  the  students  by  course  and  selected  demographic 
characteristics.  The  average  age  of  all  the  students  was  33.37  years.  As  an  approximate  estimate 
of  the  numbers  of  students  who  were  managers  versus  those  who  were  first  line  supervisors  and 
enlisted  personnel,  a  grade  of  E5  was  selected;  63.18%  of  the  students  were  E5s  or  below. 
Approximately  thirty  percent  of  the  students  had  a  duty  position  related  to  the  course  in  which 
they  were  enrolled  and  4.9%  had  a  civilian  occupation  related  to  the  course  content.  All  of  the 
students  were  high  school  graduates  or  the  equivalent;  15%  had  a  four-year  college  degree  or 
more.  Students  were  asked  how  interested  they  were  in  the  course  content  prior  to  taking  the 
course.  Approximately  seventy-three  percent  rated  themselves  as  very  interested  (5  on  a  5-point 
scale).  When  asked  if  they  had  ever  taken  a  course  taught  by  television,  18.86%  reported  that 
they  had. 


Evaluation  Instruments 

There  were  40  different  data  gathering  instruments  developed  by  FTP  personnel  in 
conjunction  with  Systems  for  Training  and  Applied  Research,  Inc.,  Lexington,  Kentucky.  In 
addition,  six  Army  forms  were  used  to  collect  the  test  data.  Appendix  A  lists  each  form,  the  data 
source  (e.g.,  student,  VTT  instructor),  the  data  collection  schedule,  and  the  course(s)  for  which 
the  data  were  collected.  While  there  are  32  forms  listed  that  were  developed  by  FTP  personnel, 
there  were  different  versions  of  Forms  Cl  and  C2  (the  pretest  and  posttests)  for  each  of  the  five 
courses.  Forms  R  and  S  were  developed,  but  these  data  were  never  collected  due  to  a  DITRA 
directive.  A  brief  description  of  the  instruments  is  provided  in  Appendix  B. 
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Table  2 


Enrollment  in  Army  MOS  Courses 
By  Component  and  Site 
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Florida  Army  National  Guard 

15 

15 
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39 

United  States  Army  Reserve 
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21 

60 

Total 

35 

34 

30 
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Enrollment  in  Hazardous  Waste 
By  Service,  Component,  Site 
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United  States  Navy 
(Military) 

18 

2 
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0 

21 

United  States  Navy 
(Civilian) 
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16 

0 

0 

0 

16 

United  States  Coast 
Guard 

0 

0 

8 

0 

0 

8 

United  State  Marine 
Corps 

0 

6 

2 

2 

0 

10 

Florida  Army 

National  Guard 

14 

8 

14 

0 

0 

36 

Alabama  Army 

National  Guard 

0 

0 

0 

0 

13 

13 

Rhode  Island  Army 
National  Guard 

0 

0 

0 

12 

0 

12 

Total 

32 

32 

24 

15 

13 

116 

Enrollment  in  Total  Quality  Leadership 
By  Service,  Component,  Site 
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United  States  Marine  Corps 
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Florida  Army  National  Guard 
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0 

37 

Florida  Air  National  Guard 

2 
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Rhode  Island  Air  National  Guard 
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0 
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Total 

22 

18 
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11 
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Table  3 
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L  Characteristics  of  Students  by  Course 
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Number  of  students 
in  course 

33 

40 

26 

116 

60 

275 

Average  age 

30.96 

32.25 

28.5 

35.5 

39.64 

3337 

Highest  grade 
(E5  or  below) 

91.2% 

62.5% 

92.3% 

34.2% 

35.7% 

63.18% 

Duty  position  related 
to  course  content 

51.4% 

37.5% 

30.8% 

12.3% 

18.6% 

Civilian  occupation 

1  related  to  course 
content 

14.3% 

2.5% 

7.7% 

NA 

NA 

4.9 

Civilian  education 

4  year  college  degree 
or  more 

11.5% 

10.0% 

0% 

6.1% 

47.4% 

15% 

Interest  in  content 
(5  point  scale  - 
Very  Interested 

85.7% 

77.5% 

80.8% 

59.6% 

62.7% 

73.26% 

Course  taught  by  TV 
|  (%  yes) 

17.1% 

12.5% 

19.2% 

16.7% 

28.8% 

18.86% 
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Items  from  the  instruments  are  included  as  necessary  when  the  results  are  reported. 
Typical  items  include  the  following: 

•  Dichotomous  data,  usually  "yes/no"  responses  or  "like/did  not  like,"  for  example. 

If  you  had  the  opportunity  to  take  additional  military  teletraining  instruction  in 
the  future,  would  you  want  to?  _ yes  _ no 

•  Ratings  on  a  3-  or  5-point  Likert  scale,  where  the  highest  number  is  always  the 
most  positive  response,  for  example. 

Please  rate  the  aspects  of  this  course  on  a  scale  from  1  to  5  or  mark  NA  if  the 
item  is  not  applicable  to  this  course: 


5  =  Excellent 
4  =  Very  Good 
3  =  Good 
2  =  Below  Average 
1  =  Poor 

NA  a  Not  Applicable 


1.  The  VTT  instructor’s  poise, 
personality  and  enthusiasm. 


1  2  3  4  5  NA 


2. 


The  VTT  instructor’s  delivery  of  1  2  3  4  5  NA 

information  over  the  network. 


Ranking  a  series  of  responses  (coded  so  the  highest  number  was  always  the  most 
positive),  for  example, 


If  you  could  pick  from  the  following  military  training  options  in  the  future,  which 
would  you  prefer  as  a  method  to  receive  training? 

_  Traditional  classroom  instruction  at  a  military  school 

_  Video  teletraining  at  a  community  college  site 

_  Correspondence  study 

_  Training  provided  locally  by  an  assigned  military  instructor 

_  Video  teletraining  received  at  your  armory/reserve  center 


Open-ended  questions,  for  example, 


If  you  could  have  changed  something  in  the  course,  what  would  it  be? 
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Procedures 

A  complete  set  of  evaluation  forms  for  each  participant  (students  and  remote  site 
personnel)  in  each  of  the  MOS  courses  were  compiled  into  a  data  collection  notebook  and 
distributed  to  each  remote  site  on  the  first  day  of  each  course.  (Notebooks  were  not  needed  for 
the  special  topics  courses  because  all  the  data  were  collected  during  these  one  day  courses). 
Directions  that  specified  what  data  were  to  be  collected,  when  it  was  to  be  collected,  and  from 
whom,  were  provided  with  the  notebooks. 

All  student  and  instructional  personnel  interviews  were  conducted  by  the  project 
evaluators.  Interviews  were  conducted  and  questionnaire  data  collected  according  to  the  schedule 
listed  in  Appendix  A.  Specific  times  were  set  aside  at  the  beginning  and  end  of  each  course  for 
data  collection  and  these  times  were  specified  on  the  training  schedule  for  each  course. 
Individual  student  interviews  were  conducted  by  the  project  evaluators  immediately  after  the 
students  completed  the  quesiionnaire(s).  (Due  to  unforeseen  circumstances.  Camp  Fogerty  was 
closed  for  the  day  before  the  TQL  course  was  completed;  this  resulted  in  no  student  data  being 
collected  at  that  site  for  that  course). 

The  data  were  organized  by  categorizing  each  item  on  each  evaluation  form  according  to 
its  related  objective  or  objectives.  The  data  were  further  organized  by  determining  whether  each 
item  provided  data  that  was  course,  site,  or  student  related.  Broad  issues  related  to  each  objective 
were  also  identified.  This  resulted  in  a  data  matrix  for  each  evaluation  objective. 

Data  Analysis 

The  data  were  coded  as  necessary  and  entered  into  a  database.  The  Windows  version  of 
the  Statistical  Package  for  Social  Sciences  (SPSS  for  Windows  5.0)  was  used  to  analyze  the  data 


Cost  Analysis 

An  analysis  of  the  cost  and  a  discussion  of  the  results  is  included  in  a  separate  section 
of  this  report. 


OBJECTIVES  RELATED  TO  THE  TECHNOLOGY 
Network  Description/Configuration 

TNET,  the  U.S.  Army's  Teletraining  Network,  was  selected  as  the  communications 
technology  for  the  FTP.  The  Network  Control  Center  (NCC)  for  TNET  is  located  at  the  U.S. 
Army  Training  Support  Center  (ATSC),  Ft  Eustis,  Virginia.  TNET  is  a  two-way  audio-video 
transmission  medium  using  Compression  Labs,  Inc.  (CLI)  Gallery™  235  VTT  systems  and 
Hughes  Network  Services  (HNS)  satellite  communications. 

The  Gallery  235  is  a  stand-alone,  modular  video  conferencing  system  that  houses  much 
of  the  equipment  (Figure  3  shows  TNET  from  the  instructor’s  perspective;  Figure  4  shows  TNET 
from  the  classroom/student  perspective).  The  TNET  system  consists  of: 

•  an  antenna  and  feedhom  (located  on  a  satellite  dish  or  very  small  aperture 
terminal  (VSAT)) 

•  an  IBM  PC  dedicated  to  communication 

•  two  35-inch  television  monitors  (one  for  viewing  motion,  one  for  graphics) 

•  a  remotely  controlled  video  camera  mounted  in  the  Gallery 

•  a  unit  for  encoding  and  decoding  the  compressed  digitized  signal 
(CODEC)  (CLI’s  Rembrandt  11/06™) 

•  an  indoor  unit  (IDU)  to  receive  and  process  the  satellite  signal 

•  a  desk-top  control  unit  to  handle  transmission  and  reception 

•  audio  equipment  including  table-top  microphones  for  student  use 

•  a  multi-media  (graphics)  computer  (origination  site  only) 

•  a  graphics  stand  with  camera  (ELMO),  and 

•  two  uninterrupted  power  supply  (UPS)  units. 

The  HNS  TELEconference™  System  provides  a  flexible  facility  for  VTT  using  a 
permanently  assigned  satellite  channel  for  management  and  a  shared  pool  of  satellite  channels 
scheduled  to  support  video  and  audio  communications  among  its  users.  The  compressed  and 
digitized  signal  is  sent  to  SBS-5,  a  geosynchronous  Ku-band  satellite  owned  by  HNS  located 
above  the  Pacific  Ocean  near  Hawaii.  The  digital  transmission  rate  used  was  256  kilobits  per 
second  (kbps). 
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TNET  communication  capabilities  include: 

•  two-way  video 

•  two-way  audio 

•  point  to  multi-point  communications 

•  year-round,  24  hour-a-day  availability 

•  interactivity  among  a  maximum  of  eight  sites  (1  origination,  7  receiving). 

During  a  teletraining  conference,  audio  and  visual  information  is  exchanged  between  two 
or  more  sites.  Audio  information  is  provided  from  microphones  that  pick  up  voice  signals  in  the 
room.  Visual  information  is  provided  from  the  motion  camera  mounted  in  the  Gallery,  an 
auxiliary  camera  such  as  that  on  the  ELMO,  computer  generated  graphics,  or  from  some  other 
source  such  as  videotape.  The  audio  and  visual  information,  originally  in  the  form  of  analog 
signals,  must  be  converted  to  digital  form  (i.e.,  digitized)  in  order  to  be  compressed  into  a  single 
signal  and  transmitted. 

The  two-monitor  system  allows  motion  to  be  seen  on  one  monitor,  graphics  on  the  other. 
However,  only  a  single  visual  signal  may  be  transmitted  each  way  at  any  given  time  due  to  the 
channel  size.  When  a  graphic  image  is  sent,  the  motion  transmission  is  frozen  for  approximately 
four  seconds.  Motion  resumes  upon  completion  of  the  transmission  of  the  graphic  image. 

Since  all  sites  are  identically  configured,  any  site  can  serve  as  the  origination  site.  To 
change  the  point  of  origination  from  one  site  to  another,  a  procedure  called  "passing  the  baton" 
is  performed  using  the  IBM  PC.  It  takes  several  seconds  to  pass  the  baton,  but  the  procedure 
is  useful  if  another  site  needs  to  become  the  origination  site  for  an  instructional  activity. 

The  technical  requirements  for  TNET  are  presented  in  the  TNET  Handbook ,  distributed 
by  the  NCC  at  Ft  Eustis,  Virginia  when  the  equipment  is  installed  at  each  site.  This  handbook 
provides  a  complete  description  of  the  system  configuration  and  features,  the  system  components, 
how  to  operate  a  teletraining  conference,  how  to  program  the  CODEC,  and  a  set  of 
troubleshooting  procedures.  NCC  also  specifies  standard  classroom  equipment,  room  size  and 
configuration,  and  room  conditions  for  TNET  use. 

Network  Guidelines 

The  number  of  sites,  number  of  students  at  each  site,  and  the  complexity  of  the  equipment 
necessitated  that  the  project  develop  a  set  of  procedures  and  guidelines  that  would  insure  the 
instruction  was  implemented  as  it  was  designed.  For  example,  a  maximum  number  of  fifteen  and 
a  minimum  number  of  nine  students  was  established  for  each  site  to  insure  that  the  instruction 
was  interactive  and  to  accommodate  the  TNET  equipment. 
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During  instruction,  use  of  the  TNET  equipment  was  coordinated  through  the  Network 
Manager  and  Chief  Engineer  at  FCCJ  who  in  turn  coordinated  with  NCC  at  Ft  Eustis.  The 
guidelines  established  for  network  use  during  the  project  are  listed  in  Appendix  C. 

Findings  from  TNET  Studies 

Several  comprehensive  studies  have  been  conducted  using  TNET.  A  number  of  these 
have  been  conducted  in  conjunction  with  the  Defense  Language  Institute  Foreign  Language 
Center  (DLEFLC)  (Bramble,  1990;  Bramble  &  Bauer,  1991,  1992).  For  example,  DLIFLC 
investigated  the  use  of  computer  assisted  study  (CAS)  and  VTT  to  address  the  language  training 
needs  of  practicing  military  intelligence  (MI)  linguists.  Three  types  of  training  technology  were 
employed:  CAS,  VTT,  and  combined  CAS/VTT.  Bramble  and  Bauer  (1992)  made  the  following 
general  conclusion,  "the  pilot  tests  demonstrated  the  potential  of  the  CAS  and  VTT  technologies 
to  provide  outstanding  resources  for  on-site  training  for  military  linguists"  (p.  103). 

The  findings  from  the  DLIFLC  German  VTT  pilot  test  (Bramble  &  Bauer,  1991)  are 
indicative  of  the  general  findings  regarding  TNET  when  it  is  used  for  linguists  training: 

•  VTT  courses  must  be  carefully  preplanned  and  designed. 

•  Adequate  training  for  site  facilitators  and  instructors  was  necessary.  This  included 
both  training  in  the  technology,  in  new  instructional  techniques,  and  for  on-camera 
teaching. 

•  The  equipment  and  communications  links  were  sufficient  to  support  VTT  training. 
However,  contingencies  were  needed  in  the  event  of  equipment  failures. 

•  Army  National  Guard  MI  linguists  met  the  instructional  objectives. 

»  Participants  were  highly  satisfied  with  the  superior  quality  of  the  course  and  the 
instructors.  They  judged  the  course  to  be  interesting  and  motivating. 

•  The  media  available  with  the  system  provided  an  excellent  media  mix  for 
language  instruction. 

•  The  accessibility  of  the  training  to  students  was  high. 

•  VTT  has  the  potential  to  support  nonresident  military  language  training. 

Another  major  study  involved  TNET  instruction  for  the  Basic  Non-commissioned  Officer 
Course  (BNCOC)  Common  Leader  Training.  This  course  was  designed  and  delivered  to  the 
Kentucky  Army  National  Guard.  The  purpose  of  the  study  was  to  determine  if  VTT  was  as 
effective  as  traditional  instruction  for  the  RCs.  The  course  was  designed  using  the  word  picture 
approach  (Cyrs  &  Smith,  1990).  One  goal  of  the  program  was  to  foster  interaction  between  the 
instructor  and  students  and  to  use  a  variety  of  interactive  and  highly  participative  techniques,  such 
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as  group  exercises  and  role  plays.  The  POI  and  tests  were  the  same  for  all  the  groups.  TRADOC 
(1992)  concluded,  "VTl  worked,  students  liked  VTT,  instructors  like  VTT,  VTT  test  scores  were 
as  good  if  not  better  than  traditional  test  scores,  and  reconfigured  materials  produced  higher 
posttest  scores  than  traditional  materials." 

Four  of  the  objectives  of  the  FTP  were  directly  related  to  the  VTT  technology  itself. 
These  included  determining  its  reliability,  determining  the  military’s  acceptance  of  the  VTT 
approach,  determining  the  success  of  the  technology  in  providing  a  viable  means  for  students  to 
access  the  training,  and  determining  how  appropriate  the  VTT  system  was  for  providing  the 
courses.  Each  of  these  objectives  is  discussed  in  detail  in  the  following  sections  along  with  a 
discussion  of  the  results  from  the  data  collected. 
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Objective  A1 

Determine  the  Reliability  of  the  Equipment  and  Communications 
Links  Required  for  the  VTT  Courses 

An  important  concern  for  the  DoD  is  whether  or  not  the  TNET  system  is  reliable  enough 
to  provide  a  viable  means  of  effectively  conducting  VTT  courses.  To  determine  this,  data  were 
collected  and  analyzed  from  technicians’  logs  and  from  questionnaires  presented  to  students  and 
instructors  regarding  their  opinions  of  the  effectiveness  of  the  technical  equipment. 

Data  from  Technicians’  Logs 

Site  technicians  were  asked  to  keep  a  log  of  each  equipment  disruption  or  outage  that 
occurred  during  the  training  sessions.  A  disruption  was  defined  as  a  breakup  or  interference  in 
the  video  or  in  the  audio.  For  example,  some  disruptions  were  as  minor  as  static  appearing  on 
the  video  display.  These  disruptions  lasted  only  a  few  seconds  at  a  time  and  in  most  cases 
resulted  in  only  minor  distractions  to  class  instruction. 

An  outage  was  described  as  a  loss  in  both  the  audio  and  video  transmissions.  Thus,  an 
outage  was  considered  to  be  more  severe  than  a  disruption.  Outages  could  occur  for  any  length 
of  time  (e.g.,  a  minute,  hour,  or  day). 

In  the  422.5  hours  of  transmission  time,  transmission  loss  occurred  for  a  total  of  1.63 
hours.  This  indicates  that  the  TNET  system  was  over  99%  reliable  for  all  five  courses.  A  series 
of  contingency  plans  had  been  established  to  insure  that  instruction  was  continuous  and  effective 
in  the  event  of  any  outages  not  under  project  control.  These  were  published  in  the  Instructor’s 
Guide  (IG)  and  the  Instructional  Coordinator’s  Guide  (ICG).  In  the  event  of  a  failure  of  any 
kind,  the  Instructional  Coordinator  (IC)  at  each  site  knew  how  to  continue  the  instruction. 
However,  due  to  the  reliability  of  the  system,  these  contingency  plans  were  rarely  used.  Table 
4  describes  the  number  and  duration  of  outages  which  occurred  during  each  course. 

Table  4 

Frequency  and  Duration  of  TNET  Outages 


TQL 

Totals 


14.75 


422.50 


1.63 


136.50 


109.00 


47.25 


76Y10 


95B10 


HAZ 
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Transmission  disruptions  were  also  recorded  in  the  technician  logs.  As  described 
previously,  these  disruptions  involved  interference  in  the  video  and/or  audio  signal,  but  did  not 
result  in  the  loss  of  video  or  audio.  They  were  categorized  according  to  their  causes  which 
included:  weather,  loss  of  power,  equipment  failure,  personnel  or  human  error,  audio  signal 
interruption,  or  video  signal  interruption.  Out  of  30  total  disruptions,  16  (52%)  involved 
problems  with  the  video.  The  majority  of  these  were  easily  corrected  by  boosting  the  video 
transmission  signal.  They  had  a  negligible  effect  on  class  instruction.  Figure  5  illustrates  the 
frequency  of  each  of  these  categories  of  disruptions. 


Figure  5.  Frequency  of  transmission  disruption  causes 


Data  from  Students  and  Instructors 

In  addition  to  the  technicians  logs,  students  were  asked  to  rate  the  quality  of  the  audio 
and  video  in  each  course  using  a  5-point  Likert  scale.  Table  5  provides  the  means.  All 
means  show  that  the  perceived  quality  of  the  audio  and  video  was  very  good  (4.0). 


Table  5 

Means  of  Students*  Perceived  Quality  of  Audio  and  Video 
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In  addition,  students  in  the  MOS  courses  were  asked  whether  they  felt  the  technical 
equipment  functioned  as  effectively  as  they  would  have  liked.  Figure  6  indicates  the 
percentage  of  students  who  responded  yes  to  the  question. 


Courea 


Figure  6.  Percentage  of  positive  student  responses  concerning  the  technical  efficiency 
of  the  equipment 


Instructors  of  the  MOS  courses  were  also  polled  as  to  their  opinions  regarding  the  quality 
of  the  audio  and  video.  Ratings  were  obtained  using  the  same  5-point  scale  used  by  the  students. 
The  mean  response  for  the  technical  quality  of  the  audio  was  4.0  which  indicates  that  the 
instructors  felt  the  audio  was  very  good.  The  mean  response  for  the  technical  quality  of  the 
video  was  4.33,  again  indicating  the  instructors  gave  the  video  quality  very  favorable  marks. 

In  summary,  the  data  from  the  technicians’  logs  and  the  responses  from  students  and 
instructional  personnel  indicated  that  TNET  functioned  very  effectively.  The  reliability  of  the 
equipment  is  important  to  the  success  of  any  VTT  program  and  TNET  was  a  reliable  system 
during  the  FTP. 
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Objective  A2 

Determine  the  Target  Communities*  Acceptance  of  the 
V1T  Approach  to  Course  Delivery 

The  purpose  of  this  objective  was  to  measure  the  reactions  of  service  members  regarding 
the  use  of  VTT  technology  to  deliver  the  instruction.  This  objective  was  defined  as  the  students’ 
acceptance  of  technology  rather  than  acceptance  of  the  teaching  and  learning  activities  associated 
with  VTT. 

Data  from  Students 

A  variety  of  questions  were  asked  to  determine  the  students’  feelings  toward  the 
effectiveness  of  the  technology  and  their  perceptions  of  the  effect  of  distance  between  the 
instructor  and  students  at  the  remote  sites.  In  addition  to  measuring  student  reactions  to  the 
technology,  questions  related  to  the  students’  acceptance  of  VTT  technology  were  also  posed  to 
the  MIAs  and  the  MSCs. 

Students  were  asked  to  rank  order  their  preferred  method  of  receiving  instruction.  Figure 
7  illustrates  the  mean  results  from  this  question.  Instruction  via  VTT  at  a  community  college  had 
the  highest  mean  of  any  other  method  of  instruction,  ranging  from  3.58  in  76Y10  to  4.01  in 
HazWaste.  VTT  instruction  at  an  armory  or  reserve  center  had  the  second  highest  ranking. 

Students  were  also  asked  an  open-ended  question  about  what  they  liked  best  about  VTT 
instruction.  Of  the  79%  of  the  students  in  all  courses  who  responded  to  this  open-ended 
question,  23%  stated  that  the  technology  was  the  aspect  of  the  instruction  that  they  liked  the  best. 
Student  responses  fell  into  six  broad  categories  in  terms  of  what  they  liked  best:  the  instructor, 
the  community  college  atmosphere,  the  technology,  the  course  materials,  the  travel,  and  other. 

When  asked  an  open-ended  question  about  what  they  liked  least  about  this  form  of 
instruction,  79%  of  the  students  responded.  The  percentage  who  indicated  that  technology  was 
the  aspect  of  the  course  that  they  liked  the  least  ranged  from  0%  in  the  95B10  course  to  3%  in 
the  71L10  course.  Thus,  the  data  indicate  that  students  were  very  positive  regarding  the 
technology  involved  with  VTT  instruction. 
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Figure  7.  Student  Rankings  of  Possible  Training  Options 


Students  were  also  asked  whether  they  believed  their  class  was  as  effective  as  a  live  class 
not  delivered  by  VTT.  In  4  of  the  5  courses,  over  75%  stated  that  the  VTT  instruction  was  as 
effective.  The  percentage  of  students  responding  yes  ranged  from  54.8%  in  71L10  to  84.1%  in 
the  HazWaste  course  (Figure  8). 


71L10  76Y10  95B10 

N-31  N.40  N-26 


Ym  54.9% 


Figure  8.  Student  Acceptability  of  Approach 


Students  were  also  asked  how  they  felt  about  the  physical  distance  separating  them  from 
the  VTT  instructor.  Since  physical  distance  between  the  student  and  the  instructor  is  a 
characteristic  of  all  distance  learning  technologies,  measuring  the  students’  responses  to  this  item 
is,  in  part,  measuring  the  students’  acceptance  of  the  technology.  The  percentage  who  perceived 
distance  as  a  problem  ranged  from  2.2%  in  the  HazWaste  course  to  21.27%  in  the  71L10  course 
(Figure  9). 
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Figure  9.  Students  Reporting  Problems  with  Physical  Distance 


When  asked  to  select  from  a  5-point  Likert  scale  the  rating  that  best  reflected  then- 
attitudes  toward  taking  another  military  course  by  teletraining,  student  responses  ranged  from  a 
mean  of  3.78  in  the  71L10  course  to  a  mean  of  4.54  in  the  TQL  course  (Figure  10).  In  a  related 
question,  posed  only  to  students  taking  the  TQL  course,  students  were  asked  if  they  would 
encourage  others  to  take  a  course  by  VTT.  Thirty-seven  students  were  asked  if  they  would 
recommend  a  VTT  course  to  a  friend,  with  86.5%  indicating  that  they  would. 
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Figure  10.  Students’  Attitudes  Toward  Additional  VTT  Courses 
Data  from  Instructional  Personnel 

The  MIAs  at  the  FCCJ  origination  site  were  asked  whether  they  would  like  to  teach  more 
classes  by  satellite  and  VTT.  Two  of  the  three  MIAs  responded  to  this  question  by  stating 
maybe.  One  MIA  responded  by  saying  yes.  All  three  MIAs  cited  non-technical  issues,  such  as 
too  many  visitors  during  transmission  and  high  stress  levels  due  to  mistakes  in  the  proponent 
school  supplied  materials,  as  drawbacks  to  VTT  teaching. 

The  nine  MSCs  in  the  Army  MOS  courses  (3  at  each  site)  were  asked  their  opinions  of 
the  students’  perceptions  of  the  training.  The  purpose  of  the  question  was  to  decide  if  the 
students  had  reported  problems  to  the  MSCs  that  they  did  not  share  with  the  civilian  evaluators 
either  on  questionnaires  or  in  personal  interviews.  Six  of  the  MSCs  reported  that  the  students 
had  positive  feelings  toward  the  VTT  approach,  but  they  also  cited  some  problems  that  the 
students  experienced  adjusting  to  the  technology.  Positive  narrative  comments  ranged  from  "The 
students  forgot  it  was  a  box  after  awhile,"  to  "While  the  students  were  impressed  (by  the 
technology),  viewing  VTT  caused  eyestrain."  Three  of  the  MSCs  reported  that  they  did  not  feel 
the  students  reacted  positively  to  the  technology.  One  MSC  in  the  95B10  course  stated  that  the 
"students  were  bored  staring  at  the  screen." 

In  summary,  both  students  and  the  instructional  personnel  accepted  the  technology. 
Students  felt  VTT  was  as  effective  as  a  live  class  and  that  the  distance  separating  students  and 
instructors  did  not  detract  from  the  instruction.  Likewise,  the  instructional  personnel  were 
generally  positive  about  the  technology  even  though  several  of  them  reported  that  care  must  be 
taken  to  insure  that  the  instruction  is  not  boring. 
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Objective  A3 

Determine  the  Success  of  the  Technology  in  Providing  a  Viable  Means 
for  Students  to  Access  the  Training 

One  objective  of  the  FTP  was  to  determine  if  teletraining  could  increase  the  availability 
of  the  military  courses  to  the  student  population.  The  benefits  that  occur  as  a  result  of  the 
increased  accessibility  of  the  courses  must  also  be  weighed  against  the  costs  incurred  while 
establishing  a  teletraining  course.  Data  were  collected  from  students  regarding  the  distance  that 
they  had  to  travel  to  reach  the  teletraining  site,  and  whether  they  commuted  daily  to  the  course, 
or  were  locally  housed  for  the  duration  of  the  course. 

Data  from  Students 

According  to  the  Joint  Travel  Regulation  (JTR),  students  who  live  farther  than  fifty  miles 
from  a  course  site  are  eligible  to  receive  billeting  and  meals  or  per  diem  throughout  the  duration 
of  each  course  (students  living  less  than  50  miles  away  were  eligible  to  receive  per  diem  for 
meals  only).  Thus,  if  teletraining  increases  the  accessibility  of  courses  to  students,  then  more 
students  should  be  able  to  commute  locally  rather  than  be  billeted  for  a  course  period,  resulting 
in  a  more  cost  effective  training  program.  With  this  in  mind,  each  student  was  asked  to  complete 
a  survey  that  requested  an  estimate  of  the  distance  in  miles  from  their  home  to  the  classroom  site 
and  whether  they  commuted  daily  to  the  course  site  or  had  to  be  housed  locally  for  the  duration 
of  the  course. 

The  three  MOS  courses  (71L10,  76Y10,  95B10)  were  selected  partly  because  a  large 
population  had  been  identified  as  needing  these  courses  in  the  selected  TNET  remote  site  areas. 
However,  as  the  courses  were  being  developed  for  teletraining,  this  target  population  decreased. 
The  researchers  found  that  not  enough  students  were  available  in  the  TNET  site  local  areas  when 
the  courses  were  scheduled  to  occur.  To  maintain  the  integrity  of  the  experimental  evaluation, 
it  became  necessary  for  students  from  outside  the  immediate  areas  to  travel  to  the  TNET  sites. 
In  some  cases,  students  living  near  one  TNET  site  were  required  to  commute  to  another  remote 
site  rather  than  attend  the  course  locally.  Thus,  the  accessibility  data  is  significantly  confounded. 
In  fact,  the  mean  distance  traveled  by  students  was  90.2  miles  (95B10),  164.08  miles  (71L10), 
and  215.48  (76Y10)  miles.  Accordingly,  the  mean  travel  time  for  students  was  1.72  hours 
(95B10),  2.36  hours  (71L10),  and  3.29  hours  (76Y10).  This  data  indicates  that  it  was  not 
feasible  for  the  majority  of  the  students  to  commute  daily  from  their  home  to  the  remote  site. 
These  students  had  to  be  housed  locally.  Figure  11  shows  the  percentage  of  students  who 
commuted  to  a  course  held  at  a  local  TNET  site  versus  those  who  had  to  travel  to  a  remote  site 
and  be  housed  for  a  course. 


Figure  11.  Percentage  of  students  who  commuted  daily  vs.  those  who  traveled  to 
remote  site. 


In  summary,  due  to  the  experimental  requirements  of  this  project,  the  data  for  this 
objective  were  confounded.  Accurate  conclusions  about  access  cannot  be  drawn.  Conceptually, 
however,  while  teletraining  is  believed  to  increase  the  accessibility  of  training,  it  is  not  without 
costs.  As  discussed  earlier,  this  technology  should  allow  a  decrease  in  costs  by  providing  sites 
to  which  students  can  commute.  Therefore,  there  should  be  a  decrease  in  billeting  costs  and  per 
diem  as  a  result  of  the  increased  availability  of  courses.  However,  this  savings  must  be  weighed 
against  the  significant  expenditures  that  were  incurred  to  establish  TNET  sites.  Thus,  the  cost 
effectiveness  of  this  technology  involves  comparing  the  equipment  costs  to  the  savings  that  occur 
as  a  result  of  less  travel  costs.  It  should  also  be  noted  that  VTT  technologies  come  in  many 
varieties;  costs  of  the  equipment  can  vary.  VTT  costs  may  change  if  alternate  technologies 
become  available. 
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Objective  A4 

Determine  How  Appropriate  the  VTT  System  is  for  Providing  the  Training 


The  purpose  of  this  objective  was  to  determine  how  well  suited  the  courses  chosen  for 
the  project  were  for  presentation  via  VTT.  The  appropriateness  of  each  of  the  five  courses  was 
evaluated:  (a)  based  on  review  of  current  research  regarding  course  selection  for  VTT,  (b)  by 
analyzing  additional  factors  such  as  the  number  of  off-line  activities  and  the  stability  of  the 
course  content,  and  (c)  by  student  perceptions  of  the  appropriateness  of  the  courses  for  VTT 
presentation. 

In  selecting  courses  for  VTT  delivery,  demand  (throughput)  is  often  a  significant  concern 
from  a  cost  perspective.  In  the  case  of  the  FTP,  the  military  selected  the  five  courses  primarily 
because  of  the  high  population  in  Florida  requiring  these  courses.  The  MOS  courses  were  chosen 
because  soldiers  needed  to  be  qualified  in  these  particular  MOSs  to  fulfill  important  military  roles 
and  functions  within  their  assigned  units.  In  addition,  other  factors  were  taken  into  consideration 
when  these  courses  were  selected.  These  included  the  expense  involved  in  sending  soldiers  to 
proponent  schools,  RC  soldiers’  employment  schedules,  whether  or  not  the  students’  work 
schedules  would  allow  them  to  take  an  8-10  week  course,  and  when  classes  were  scheduled  at 
the  local  USARF  schools.  National  Guard  Academies  and  the  proponent  schools  (e.g.,  some 
classes  might  not  be  offered  until  the  following  year).  Likewise,  the  Navy  considered  the 
HazWaste  and  TQL  courses  to  be  very  important;  the  content  was  needed  by  a  large  number  of 
personnel.  The  Navy  courses  were  also  selected  because  of  their  potential  interest  to  multi¬ 
service  audiences. 

The  Navy  prefers  to  present  courses  no  longer  than  one-week  in  length  via  its  teletraining 
system  (CNET  Handout,  1992).  The  three  MOS  courses  were  presented  during  a  two- week,  80- 
hour  training  period.  Although  the  MOS  courses  were  longer  than  the  Navy  recommendation, 
the  instruction  contained  numerous  off-line  activities  that  helped  to  breakup  long  periods  of  VTT 
presentation.  The  special  topics  courses,  HazWaste  and  TQL,  were  presented  as  one-day 
workshops. 

An  additional  factor  regarding  the  appropriateness  of  the  courses  for  VTT  was  the  number 
of  off-line  activities  that  could  be  designed  for  the  course.  The  fact  that  the  MOS  courses  were 
presented  during  a  two-week  block  had  the  potential  of  being  problematic  because  students  could 
not  be  expected  to  attend  to  continuously  televised  instruction  for  that  length  of  time.  However, 
two  of  the  courses,  71L10  and  95B10,  included  psychomotor  skills  (e.g.,  typing,  demonstrating 
MP  search  procedures);  all  five  courses  had  practical  exercises  (PEs)  as  required  course 
components  that  allowed  (and  sometimes  required)  students  to  go  off-line  for  hands-on  practice 
for  varying  periods  of  time.  The  FTP  course  developers  used  these  off-line  exercises  as  a  method 
to  alleviate  any  boredom  or  lack  of  attention  that  might  occur  as  a  result  of  listening  to  eight 
hours  of  instruction.  Off-line  activities  also  provided  interactivity  for  the  students,  enabled  them 
to  practice  what  they  had  learned,  and  provided  a  means  for  the  students  to  interact  with  one 
another. 

Finally,  stability  of  the  lesson  content  found  in  the  courses  was  also  a  consideration  of 
appropriateness  for  VTT  because  courses  that  have  frequent  content  changes  are  not  typically 
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good  candidates  for  VTT  (or  for  any  mediated  delivery  strategy).  This  is  because  course 
resources  can  become  outdated  too  quickly  to  make  the  design  and  development  process  cost 
effective.  Two  courses  in  the  FTP  study,  71L10  and  76Y10,  had  regulations  that  were  expected 
to  become  outdated.  To  eliminate  this  potential  problem,  the  FTP  course  developers  omitted  the 
regulation  numbers  and  pamphlet  numbers  from  the  student  guides  and  from  the  graphics  so  that 
students  could  write  in  an  updated  version  number  as  the  course  was  presented.  The  instructor 
and  the  site  facilitator  guides  had  regulation  and  pamphlet  numbers  printed  in  them  so  the 
instructors  would  have  a  ready  reference.  In  this  way,  the  graphics  could  be  reused  without 
being  out-of-date  and  students  had  a  visual  cue  that  the  regulation  numbers  may  change  (the 
regulation  number  had  to  be  written  in). 

Based  on  a  review  of  the  POIs  and  syllabi,  a  preliminary  examination  of  the  five  selected 
courses  was  performed  in  an  effort  to  determine  how  well  they  fit  five  important  guidelines  for 
course  selection.  Table  6  provides  the  results  of  this  overview. 


Table  6 

Course  Characteristics  for  VTT  Adaptability 


■  s 


71L10 

Yes 

Yes 

Yes 

Yes 

Yes 

76Y10 

Yes 

No 

Yes 

No 

Yes 

95B10 

Yes 

Yes 

Yes 

Yes 

Yes 

HAZ 

Yes 

No 

Yes 

Yes 

Yes 

TQL 

Yes 

No 

Yes 

Yes 

Yes 

Data  from  Students 


In  an  effort  to  assess  the  students’  perceptions  of  the  appropriateness  of  the  technology, 
students  were  asked  for  what  types  of  courses  they  felt  the  technology  was  most  appropriate.  The 
narrative  data  were  then  analyzed  and  categorized  according  to  commonalities.  In  the  71L10 
course,  a  total  of  twenty-five  students  responded  to  the  question.  Fourteen  of  the  respondents 
(56%)  stated  that  they  felt  VTT  was  best  suited  for  courses  that  were  administrative  in  nature  or 
dealt  with  "paperwork"  (primarily  cognitive).  Twenty  percent  of  the  students  made  comments 
that  VTT  would  not  be  good  for  courses  that  required  "hands-on"  (psychomotor)  training. 

Twenty-nine  students  in  the  76Y10  responded  to  the  question.  Fourteen  students  (48%) 
stated  that  they  felt  that  VTT  was  best  suited  for  administrative  courses.  An  additional  seven 
students  (24%)  stated  that  they  did  not  think  VTT  was  appropriate  for  courses  that  were  "hands- 
on"  in  nature. 


Students  in  the  95B10  course  had  perceptions  similar  to  those  in  the  71L10  and  76Y10 
courses.  Twenty-five  students  responded  to  the  question.  Six  of  the  students  (24%)  mentioned 
that  they  felt  VTT  was  useful  for  teaching  administrative  type  courses.  Eleven  students  (44%) 
believed  that  VTT  was  not  useful  for  teaching  "hands-on"  type  courses.  Table  7  shows  the 
results  of  the  analysis  of  the  narrative  data  relating  to  student  perceptions  of  the  appropriateness 
of  the  VTT. 


Table  7 

Student  Perceptions  of  VTT  Course  Appropriateness 
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In  summary,  the  five  FTP  courses  selected  for  VTT  instruction  were  assessed  as  generally 
appropriate  for  the  technology.  The  psychomotor  or  "hands-on"  aspect  of  a  course  has  been 
reported  in  the  literature  as  being  potentially  problematic  and  the  students  in  the  MOS  courses 
concurred  by  stating  their  concerns  about  "hands-on"  courses.  However,  the  psychomotor  aspect 
of  the  FTP  courses  allowed  course  developers  to  design  useful  and  appropriate  off-line  activities. 
This  was  useful  for  breaking  up  long  periods  of  instruction. 
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Conclusions  Regarding  Technology 

These  objectives  have  addressed  potential  concerns  related  to  the  technology  required  to 
deliver  VTT  courses.  The  data  collected  support  the  high  reliability  and  quality  of  the  TNET 
equipment,  finding  it  to  be  over  99%  reliable  in  terms  of  number  of  disruptions  and  outages  that 
occurred  during  this  project.  Students  and  instructors  were  apparently  pleased  with  the  quality 
of  the  audio  and  video  as  they  found  it  to  be  above  average. 

Furthermore,  based  on  the  sample  students  chosen  for  the  pilot  study,  it  would  appear  that 
there  is  a  general  acceptance  of  the  VTT  approach  within  the  target  community.  The  students 
generally  liked  the  VTT  approach  better  than  the  other  training  options  available  to  them  (e.g., 
traditional  training  at  a  military  site,  traditional  training  at  a  local  armory  or  reserve  center,  or 
correspondence  courses)  and  expressed  a  preference  for  VTT  at  a  community  college  over  VTT 
at  their  local  armory  or  reserve  center.  Students  also  indicated  that  they  would  take  additional 
VTT  courses. 

The  data  collected  in  an  effort  to  determine  whether  teletraining  provides  more 
accessibility  to  training  were  confounded  by  the  need  to  maintain  experimental  integrity  by 
requiring  a  minimum  number  of  students  per  course  site.  Therefore,  the  results  of  this  project 
cannot  be  used  to  support  or  refute  the  idea  that  teletraining  provides  more  access  to  training  for 
students.  However,  in  theory  it  should  provide  better  access. 

The  five  courses  selected  for  the  FTP  were  determined  to  be  generally  appropriate  for 
VTT  presentation  according  to  the  guidelines  outlined  in  previous  research  (McDonald,  et  al„ 
1990;  CNET  Handout,  1992).  All  five  courses  were  of  short  (two  weeks  or  less)  duration,  were 
primarily  cognitive  (although  two  courses  had  psychomotor  content),  and  provided  enough 
throughput  to  satisfy  military  needs.  Four  of  the  courses  had  relatively  stable  content  and  all 
provided  opportunities  to  break  up  the  VTT  presentation  by  using  off-line  activities.  Student 
responses  tended  to  validate  the  adapted  models  used  by  the  project  to  determine  the 
appropriateness  of  the  courses  for  VTT.  The  students  perceived  VTT  as  being  appropriate  for 
courses  dealing  with  administrative  subjects  (cognitive  tasks)  and  having  little  psychomotor 
content. 

Issues  to  Consider 

An  issue  to  consider  when  assessing  the  reliability  of  the  TNET  equipment  is  that  it  is 
sensitive  to  certain  inclement  weather  conditions  such  as  fog,  rain,  and  snow.  Such  inclement 
weather  at  a  site  location  can  result  in  transmission  being  interrupted  or  severed  entirely.  This 
is  due  to  atmospheric  disturbances  that  weaken  the  TNET  signal  being  transmitted  from  the 
satellite  before  it  reaches  the  destination  site.  Often,  this  problem  can  be  alleviated  by  boosting 
the  wattage  of  the  signal  being  uplinked  to  the  satellite;  the  resulting  reflected  signal  is  of 
sufficient  strength  to  produce  a  transmission  without  any  interference.  The  normal  TNET  power 
level  is  1.5  watts,  which  can  be  boosted  to  a  maximum  of  3  watts,  if  necessary.  However, 
serious  rain  or  snow  storms  can  result  in  transmission  being  lost  entirely.  Given  that  atmospheric 
disturbances  can  disrupt  satellite  transmission,  land  lines  may  be  considered  as  a  possible 
alternative. 
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Another  issue  related  to  equipment  reliability  is  the  role  of  the  technician  during 
teletraining.  Site  technicians  were  present  at  all  sites  during  each  of  the  five  courses.  However, 
due  to  the  high  reliability  of  the  equipment,  the  technicians’  expertise  was  rarely  needed  except 
at  system  startup  and  shutdown.  Each  site  was  also  provided  with  troubleshooting  procedures 
for  audio  and  video  problems  as  well  as  guidelines  for  starting  and  shutting  down  the  equipment. 
When  determining  the  personnel  required  to  provide  a  teletraining  course,  it  is  necessary  to  assess 
whether  a  technician  is  needed  full-time  or  if  someone  else  can  be  trained  in  the  startup  and 
shutdown  procedures  with  a  technician  available  for  emergencies.  During  the  delivery  of  one 
course  in  this  project,  two  of  the  sites  were  run  without  the  aid  of  technicians.  Two  1ST 
researchers  were  trained  in  operating  and  troubleshooting  procedures;  they  also  attended  a 
workshop  on  TNET.  While  they  did  not  have  the  expertise  of  the  site  technicians,  they  were  able 
to  operate  the  equipment  effectively.  However,  they  felt  that  it  was  necessary  to  have  a 
technician  available  in  the  event  they  had  a  question.  In  one  instance,  the  researcher  was  able 
to  telephone  a  technician  at  the  NCC  in  Ft  Eustis  who  then  provided  instructions  on  how  to 
resolve  a  problem.  This  was  accomplished  in  a  matter  of  minutes.  This  problem  occurred  during 
startup,  thirty  minutes  before  class,  so  no  disruption  of  instruction  occurred.  However,  in  the 
event  of  a  major  outage,  a  technician  might  be  necessary.  The  trade-off  between  hiring  a 
technician  and  accepting  the  possibility  of  an  outage  is  an  issue  that  should  be  taken  into 
consideration  when  planning  to  use  any  VTT  equipment.  Instructional  contingencies  may  be 
effective  in  some  instances  and  should  be  included  in  teletraining  course  planning. 

Another  issue  that  must  be  considered  when  determining  the  viability  of  teletraining  as 
it  relates  to  equipment  reliability  is:  how  long  can  a  system  outage  occur  before  the  lost 
instruction  becomes  irretrievable?  This  question  was  not  answered  by  the  current  study,  as  none 
of  the  outages  that  occurred  had  a  major  impact  on  the  instruction.  AETD  was  asked  this 
question  and  they  were  unable  to  give  a  definitive  answer.  While  it  is  expected  that  a  complete 
loss  of  instruction  would  be  a  rare  occurrence,  it  might  be  necessary  to  develop  contingency  plans 
to  address  this  issue. 

When  addressing  the  acceptance  of  VTT,  one  issue  to  consider  is  whether  acceptance  is 
a  result  of  the  novelty  of  the  technology.  Clark  and  Salomon  (1986)  have  observed  that  new 
technologies  are  likely  to  be  more  acceptable  initially  because  they  generally  provide  better 
prepared  instructional  materials  and  the  novelty  of  the  technology  tends  to  better  engage  student 
interest. 

Student  acceptance  is  only  one  hurdle  that  VTT  must  overcome  to  be  a  viable  military 
training  option.  How  the  VTT  approach  fits  into  the  overall  training  system  is  an  issue  of  much 
concern  to  the  training  community.  During  the  instructional  personnel  training  sessions  held  in 
Jacksonville,  HSARF  personnel  asked  several  questions  such  as  "What  will  happen  to  the  USARF 
schools  if  the  Army  adopts  VTT  as  the  primary  means  of  reclassifying  RC  soldiers?"  and  "What 
will  happen  to  the  USARF  school  personnel  if  the  schools  are  deactivated  because  civilian 
institutions  are  conducting  the  training  traditionally  delivered  by  the  USARF  schools?"  The 
importance  of  these  questions  is  that  many  personnel  involved  in  formal  individual  training  at 
Reserve  Component  Training  Institutions  (RCTIs)  (USARF  schools.  National  Guard  Military 
Academies)  perceive  VTT  as  a  threat  to  their  positions  and  their  institutions.  To  foster 
acceptance  of  VTT  among  trainers  and  leaders  in  the  military,  plans  must  be  formulated  to  reduce 
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organizational  resistance  to  the  institutionalization  of  VTT.  These  programs  should  focus  on  the 
benefits  of  VTT  to  service  members  and  leaders,  while  minimizing  fears  regarding  "rice  bowl" 
issues. 


In  a  letter  to  the  FTP  Manager,  the  Adjutant  General  of  Florida  provided  this  assessment 
of  the  project: 

"Community  college  instructor  support  appears  redundant  and  ineffective  in  that 
USARF  instructors  were  assigned  at  all  locations  to  handle  student  questions  and 
concerns  that  the  civilian  instructors  did  not  have  the  depth  of  knowledge  to 
handle.  Proponent  school  instructors  or  USARF  instructor  personnel,  with  proper 
technological  training,  can  provide  in  depth  instruction  using  the  Army  TNET 
system  already  in  place.  The  location  of  downlink  classrooms  at  local  community 
colleges  is  not  essential  to  the  success  of  teletraining.  Equipment  placed  in  unit 
armories  and  at  the  Camp  Blanding  Training  Site  would  be  more  cost  effective 
and  more  readily  available  to  our  soldiers." 

To  summarize  this  concern,  the  purpose  of  VTT  needs  to  be  formally  established  in 
relation  to  other  training  options,  e.g.,  how  will  VTT  fit  with  other  training  options?  What  types 
of  courses  will  be  taught  using  VTT?  Instructors  and  other  RCTI  personnel  should  be  informed 
that  they  can  also  be  trained  to  deliver  VTT  instruction. 

When  selecting  courses  to  be  taught  using  this  technology,  courses  must  be  chosen 
considering  the  projected  student  populations  in  the  remote  sites.  This  was  a  lesson  learned 
during  the  FTP  where  the  courses  were  selected  considering  the  population  existing  at  the  time 
of  project  startup  without  projecting  the  population  at  the  time  of  the  implementation.  The  larger 
the  student  population  that  can  be  served  using  teletraining,  the  greater  the  potential  cost  savings. 
An  on-going  VTT  program  might  eliminate  this  problem. 

Finally,  the  selection  of  a  course  containing  some  psychomotor  components  for  VTT  may, 
in  fact,  be  useful  because  the  off-line  activities  may  serve  to  make  the  instruction  more 
interesting  by  breaking  up  long  periods  of  lecture/conference.  However,  the  inclusion  of  off-line 
activities  may  be  an  inefficient  use  of  limited  satellite  time.  Conducting  off-line  activities  may 
cause  scheduling  problems  for  large  scale  networks  with  many  different  types  of  courses  being 
delivered  across  several  time  zones.  New  instructional  strategies  and  learning  activities  must  be 
developed  to  break-up  long  periods  of  VTT  instruction  in  courses  where  off-line  activities  are 
inappropriate.  Courses  that  have  shorter  daily  instructional  periods  (2  hours  per  day  as  opposed 
to  8  hours)  may  not  need  extensive  off-line  instructional  activities. 
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OBJECTIVES  RELATED  TO  INSTRUCTION 
Course  Reconfiguration  Process 

The  five  courses  selected  for  VTT  delivery  had  to  be  converted  from  the  standard  mode 
of  platform  delivery  to  VTT  delivery.  The  five-component  SAT  model  was  adapted  for  use  in 
reconfiguring  the  courseware.  In  the  adapted  model,  th?  five  functions  of  the  SAT  were  included 
(i.e..  Analysis,  Design,  Development,  Implementation,  and  Evaluation)  and  two  functions  were 
added.  Revise  Instruction  and  Management.  The  adapted  model  is  based  on  learning  and 
instructional  theory  and  is  presented  in  Figure  12  with  some  associated  tasks. 

The  major  tasks  completed  during  each  phase  were  as  follows: 

Analysis  Phase.  This  phase  would  normally  involve  analyzing  courses  and  selecting 
those  suitable  for  presentation  over  TNET.  However,  the  five  courses  that  were  reconfigured 
during  the  FTP  were  assigned  by  the  government.  Therefore,  traditional  tasks  performed  during 
the  analysis  phase  of  the  SAT  model  were  not  conducted  during  this  project. 

Design  Phase.  During  this  phase,  the  POIs  and  syllabi  were  analyzed  to  determine  the 
adequacy  of  the  course  materials  from  an  instructional  design  perspective.  In  order  to  begin 
reconfiguring  the  courseware,  course  developers  wrote  or  modified  objectives  and  criterion- 
referenced  tests,  identified  the  conditions  of  learning  and  events  of  instruction  (Gagne,  1985), 
selected  media  mixes  for  objectives  or  groups  of  objectives,  identified  all  the  military  materials 
and  equipment  needed  to  conduct  the  instruction,  and  determined  student  entry  behaviors.  In 
addition,  global  media  and  instructional  strategy  selections  were  made  that  would  facilitate 
student  learning  and  that  matched  the  instructional  conditions  for  VTT. 

Development  Phase.  The  major  activities  of  the  Development  Phase  were  to  develop  and 
produce  the  instruction.  All  course  materials  were  developed  during  this  phase  of 
reconfiguration:  IG,  ICG,  the  ISG,  screen  graphics  and  word  pictures,  and  support  media  (e.g., 
videotapes,  charts).  In  order  to  produce  these  guides,  lesson  plans  were  developed  and 
surrounding  materials  (e.g.,  introductions  to  the  guides  and  classroom  protocols)  were  written  and 
produced.  Also  during  this  phase,  initial  pilot  testing  of  the  courseware  was  conducted  and 
validation  and  approval  of  the  courseware  from  the  responsible  military  organization  was 
obtained. 

Implementation  Phase.  Preparation  for  the  delivery  of  instruction  and  implementation 
of  the  courses  were  the  primary  goals  of  this  phase  of  the  reconfiguration  process.  The  primary 
tasks  performed  were:  acquiring  military  materials  and  equipment,  acquiring  instructional 
equipment,  scheduling  labs  and  facilities  at  the  remote  sites,  organizing  for  instruction  (e.g., 
planning  for  contingencies),  preparing  paperwork  and  recordkeeping  to  be  used  during  instruction 
(e.g.,  visitor’s  logs  and  TNET  logs),  producing  sufficient  quantities  of  course  materials,  delivering 
the  courseware  to  the  remote  sites,  preparing  the  staff  to  implement  the  instruction,  and  the  actual 
delivery  of  the  instruction. 
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Management/Organization 


42 


Revise  Instruction  Phase.  This  phase  was  added  to  the  adapted  model  because  several 
revision  and  validation  cycles  had  to  be  performed  during  the  reconfiguration  process.  The 
functions  of  this  phase  were  ongoing: 

•  During  the  Design  phase,  minor  modifications  were  made  to  the  military  POIs  and 
syllabi  to  align  the  objectives,  performance  test  items,  and  practical  exercises. 

•  During  the  Development  phase,  process  (e.g.,  readability)  and  product  (e.g., 
performance)  data  were  collected  during  several  expert  reviews  of  the  print  and 
non-print  course  materials,  and  during  trial  runs  of  some  key  instructional 
strategies.  Prototype  units  were  validated  by  the  appropriate  military  organization. 
Final  versions  of  the  course  materials  were  prepared  during  this  phase. 

•  During  the  Implementation  Phase,  a  pilot-test  site,  called  an  Intensive  Site,  was 
established  at  the  remote  site,  FCCJ2.  The  purpose  of  this  site  was  to  collect  data 
so  that  the  courses  could  be  modified  and  revised  during  delivery.  Final 
courseware  revisions  were  made  to  each  course  after  the  training  was  delivered. 

Management  Phase.  Three  primary  groups  of  people  with  a  web  of  complex 
relationships  among  them  had  to  be  coordinated  for  the  success  of  the  project.  The  project 
director  at  1ST  and  the  project  manager  at  DITRA  were  responsible  for  this  coordination.  These 
groups  included:  (a)  the  design,  development,  production,  implementation,  and  evaluation  teams, 
(b)  the  military  organizations  and  groups  involved  in  the  project,  and  (c)  the  three  community 
colleges. 

Evaluation  Phase.  A  considerable  amount  of  data  was  collected  during  delivery  of  the 
courses.  One  of  the  roles  of  project  personnel  was  to  collect  data  from  students  and  instructional 
personnel.  These  results  are  presented  in  this  document. 

Course  Delivery 

Instruction  is  presented  live  over  TNET.  At  the  origination  site,  the  primary  person 
responsible  for  content  presentation  was  the  VTT  Instructor.  The  content  of  the  courses, 
however,  required  that  the  MIA  (who  was  also  an  SME)  deliver  some  of  the  content.  Chiring 
delivery,  up  to  three  people  were  needed  to  implement  the  instruction.  While  the  VTT  instructor 
was  presenting,  it  was  necessary  for  the  MIA  to  operate  the  ELMO  because  of  his/her  content 
expertise.  A  third  person  operated  the  graphics  computer. 

At  each  remote  site,  a  community  college  site  coordinator,  who  was  also  the  IC,  was 
needed  as  the  instructor  of  record  for  each  course.  The  IC  also  performed  some  of  the  off-line 
instructional  roles.  For  the  MOS  courses,  a  MSC  was  also  required.  The  IC  and  MSC  were  the 
VTT  instructors’  representatives  at  the  remote  sites.  It  was  their  responsibility  to  manage  the 
instructional  activities  at  each  remote  site. 
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The  MSC  was  also  responsible  for  recordkeeping,  adhering  to  the  off-camera  course 
schedule,  distributing  and  maintaining  instructional  materials  (e.g.,  regulations,  pamphlets), 
evaluating  off-line  exercises,  sending  course  materials  back  to  the  VTT  instructor,  and  conferring 
with  the  on-camera  instructor.  For  the  9SB 10  course,  the  MSC  also  performed  some  instructional 
roles  and  graded  some  of  the  off-line  activities.  The  IC  performed  these  roles  during  the  one-day 
special  topics  courses.  In  these  courses,  students  were  not  being  formally  certified  so  the  military 
role  could  be  performed  by  community  college  personnel.  A  set  of  protocols  was  developed  to 
coordinate  instructional  events  during  course  delivery,  including  who  was  responsible  for 
directing  and  evaluating  military  examinations,  how  the  TNET  gated  audio  was  to  be  used,  and 
describing  the  purpose  of  the  after  action  review  (AAR). 

The  MIA  functioned  as  the  Non-Commissioned  Officer  in  Charge  (NCOIC)  for  the 
course,  and  the  MSC  functioned  as  the  NCOIC  for  each  site.  The  MIA  as  NCOIC  was  the 
overall  authority  for  course  content,  the  lead  USARF  school’s  commandant’s  representative,  and 
as  such  had  the  authority  to  grant  leave.  The  MSC  as  NCOIC  was  responsible  for  logistics  and 
personnel  functions  (e.g.,  military  pay  and  allowances,  billeting,  orders,  and  health  and  welfare 
of  the  troops)  in  addition  to  his/her  instructional  duties. 

Course  Delivery/Training  Schedules 

A  detailed  course  delivery  schedule  was  developed  for  each  course  and  included  in  the 
IG,  the  ICG,  and  the  ISG.  The  training  schedule  was  a  daily  listing  of  all  events  including  who, 
what,  when  and  where  for  all  project  and  course  related  activities.  A  sample  training  schedule 
used  for  TQL  is  included  in  Appendix  D. 
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Objective  B1 

Determine  Whether  the  Students  Met  the  Stated  Learning 
Objectives  as  a  Result  of  the  VTT  Instruction 

The  purpose  of  this  objective  was  to  provide  evidence  that  the  students  taking  the  VTT 
courses  mastered  the  training  objectives  that  were  either:  (a)  stated  in  the  POIs  of  the  MOS 
courses  or  (b)  developed  from  the  syllabi  of  the  special  topics  courses.  One  requirement  of  the 
FTP  was  that  the  exact  performance  measures  used  in  the  RC3  courseware  for  the  MOS  courses 
(71L10,  76Y10,  and  95B10)  be  used  to  assess  student  performance  during  the  VTT  instruction. 
For  the  special  topics  courses  (HazWaste  and  TQL)  the  VTT  instructors  and  MI  As  derived  the 
performance  measures  from  the  stated  learning  objectives  of  the  courses.  Since  these  two  courses 
were  adapted  from  longer  courses,  there  were  no  prepared  performance  measures  that  could  be 
used. 

Percentage  of  Students  Passing  the  Performance  Measures 

MOS  Courses.  Students  completed  the  same  performance  tests  (PTs)  that  were 
administered  to  those  taking  standard  RC3  courses.  There  were  four  performance  measures  given 
for  71L10,  six  for  76Y10,  and  35  for  95B10.  These  are  listed  in  Tables  Bl.l,  B1.2,  and  B1.3 
in  Appendix  E.  Also  included  in  these  tables  are  the  percentage  of  students  who  passed  each  PT 
on  the  first  attempt,  the  means  and  standard  deviations  of  the  students  passing  on  the  first 
attempt,  and  the  percentage  of  students  who  passed  the  PTs  on  the  retest.  A  summary  of  the 
data,  giving  the  percentage  of  students  who  passed  all  the  PTs  in  each  course  on  the  first  attempt 
is  included  in  Figure  13. 


71L10  76Y10  95B10 


Figure  13.  Average  Percentage  of  Students  Passing  All  PTs  on  the  First  Attempt  in 

MOS  Courses.  * 

*  Note:  100%  of  students  passed  the  three  MOS  courses. 
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Over  85%  of  the  students  in  the  three  MOS  courses  passed  all  the  PTs  on  the  first  attempt 
and  100%  of  the  students  passed  the  courses.  These  results  indicate  a  high  rate  of  success  for 
students  in  the  MOS  courses.  With  the  exception  of  the  students  who  had  to  retake  the  typing 
test  (this  option  was  given  to  all  students  who  did  not  master  typing  during  71L10),  all  students 
were  certified  in  the  course  they  took. 

Special  Topics  Courses.  Students  in  each  of  the  special  topics  courses  took  a  20-item 
multiple-choice  test  devised  by  the  community  college  instructor  and  the  MIA.  These  tests  were 
approved  by  the  appropriate  agency,  but  the  tests  were  not  used  for  certification.  There  were  no 
provisions  for  retesting.  The  means  and  standard  deviations  are  listed  in  Table  8.  For  the 
Haz Waste  course,  the  mean  score  was  78.23.  For  the  TQL  course,  the  mean  score  was  82.90. 

Comparison  of  Pretest  and  Posttest  Data 

Pretest  and  posttest  measures  were  administered  in  all  courses  except  TQL.  Due  primarily 
to  time  constraints,  only  selected  PTs,  or  selected  items  from  the  PTs,  were  administered  as  a 
pretest. 

For  the  71L10  course,  four  performance  measures  and  a  timed  typing  test  were  given  to 
students.  However,  only  the  first  PT,  Type  a  Memorandum,  was  completed  by  all  students.  That 
performance  measure  and  the  timed  typing  test  were  used  as  the  pretest  and  posttest  measures. 

For  the  76Y10  course,  a  50-item  achievement  test  was  used  as  the  pretest  and  post 
measures  in  lieu  of  the  PTs.  The  achievement  test  had  been  previously  developed  and  used  at 
the  3391st  USARF  by  the  MIA  for  76Y10.  The  test  was  revised  and  used  as  the  pre-post 
measure  for  this  course. 

For  the  95B10  course,  all  of  the  multiple-choice  items  administered  as  PTs  were  used  as 
the  pretest.  These  items  were  compared  to  the  same  items  taken  on  the  PTs. 

For  the  HazWaste  course,  a  20-item  test  was  used  as  the  pretest  and  posttest.  It  was 
devised  by  the  VTT  instructor  and  the  MIA. 

For  the  TQL  course,  no  pretest  was  administered.  The  pretest  was  omitted  from  the 
course  at  the  government’s  direction;  no  comparisons  can  be  made  for  this  course. 

Table  8  lists  the  pre  and  post  performance  measures  used  in  the  analysis.  A  t-test  for 
dependent  samples  was  used  to  determine  if  students  made  significant  gains  as  a  result  of  the 
instruction,  based  on  the  pretest-posttest  differences.  All  t-tests  were  significant  (p  <  .001). 
Individual  t- values  are  presented  in  Table  8.  The  comparison  of  the  pretest  and  posttest  data, 
combined  with  percentage  of  students  passing  the  performance  measures  leads  to  the  conclusion 
that  the  instruction  was  effective  in  meeting  the  learning  objectives. 
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Table  8 

Pretest  and  Posttest  Performance  Data 
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wmmm 

1  71L10 

Timed  Typing 

20.48(9.58) 

(33) 

32.42(9.58) 

(33) 

11.41* 

(32) 

Type  a  Memo¬ 
randum 
(#  errors) 

7.42(4.81) 

(33) 

1.85(1.54) 

(33) 

6.97* 

(32) 

1  76Y10 

50-item 

Achievement 

Test 

59.95(16.63) 

(40) 

76.00(11.25) 

(40) 

8.57* 

(39) 

95B10 

90-item  PTa 

71.81(10.22) 

(26) 

92.58(10.22) 

(26) 

13.56* 

(25) 

HAZ 

20-item  PT 

47.68(13.53) 

(111) 

78.23(10.95) 

(111) 

21.54* 

(HO) 

TQL 

20-item  PT 

No  pretest  given 

82.90(11.01) 

(48)b 

NA 

*£<.b6l,  one-tailed. 

*PT  =  Performance  Test 

‘’Note:  There  are  1 1  missing  scores  from  the  Rhode  Island  site. 


Self-Report  Data 


Since  performance  measures  could  not  be  obtained  for  each  student  on  each  task,  self- 
report  data  were  also  gathered.  Prior  to  the  beginning  of  each  course  (except  HazWaste), 
students  were  asked  to  rate  whether  or  not  they  could  perform  a  task  and,  in  some  cases,  how 
proficient  they  were  in  performing  the  task. 

With  the  exception  of  the  task  Type  a  Letter,  over  70%  of  the  71L10  students  reported 
that  they  had  never  done  any  of  six  tasks,  and  over  80%  reported  that  they  did  not  know  how 
to  perform  any  of  the  tasks.  Fifty-five  percent  of  the  students  reported  that  they  had  performed 
the  task  Type  a  Letter,  and  46%  reported  that  they  knew  how  to  perform  the  task.  Student  self- 
report  data  for  71L10  are  presented  in  Table  B  1.4  in  Appendix  E. 

Since  100%  of  the  students  passed  the  PT  for  these  six  tasks,  and  since  a  majority  of  the 
students  reported  that  they  could  not  perform  five  of  the  six  tasks  on  the  self-report  pretest,  these 
data  support  the  previous  conclusion  that  the  instruction  was  effective  in  meeting  the  stated 
learning  objectives. 

Students  in  76Y10  and  95B10  were  asked  to  rate  their  proficiency  on  some  or  all  of  the 
tasks  in  the  course  they  were  taking  (using  a  5-point  rating  scale).  Self-report  data  was  collected 
both  before  and  after  the  instruction. 

Table  B1.5  in  Appendix  E  presents  the  results  of  the  76Y10  student  ratings  for  seven 
different  tasks.  On  the  pretest,  the  average  student  proficiency  rating  on  seven  different  tasks 
was  2.5.  On  the  posttest,  the  average  ranking  was  4.6.  The  average  mean  score  of  the 
achievement  posttest  was  76.00  (the  Army  requires  a  minimum  score  of  70%  for  passing).  The 
consistency  of  the  students’  self-report  scores  and  their  ability  to  perform  on  the  posttest  supports 
the  effectiveness  of  the  instruction. 

Table  B1.6  in  Appendix  E  presents  the  results  of  the  95B10  student  ratings  for  11 
different  tasks.  On  the  pretest,  the  average  student  proficiency  rating  was  2.0  over  the  1 1  tasks. 
On  the  posttest,  the  average  rating  was  4.7.  The  mean  score  on  the  posttest  was  92.58.  Again, 
the  consistency  between  students’  self-reported  proficiency  and  their  ability  to  actually  perform 
on  the  selected  pretest  tasks  appears  to  support  the  effectiveness  of  the  instruction. 

Table  B1.7  in  Appendix  E  presents  the  results  of  the  TQL  student  ratings  for  four 
different  objectives.  Student  ratings  were  the  only  data  obtained  as  a  pretest  measure.  Students 
rated  their  knowledge  of  the  various  TQL  concepts  on  the  pretest  from  1.1  to  2.3  on  the  5-point 
scale.  The  mean  score  on  the  posttest  was  82.9,  thereby  supporting  the  effectiveness  of  the  VTT 
training. 

Students  in  the  MOS  courses  were  also  asked  three  additional  questions  at  the  end  of  the 
course  concerning  the  adequacy  of  the  training  they  received  in  the  skills  required  to  perform  the 
MOS.  The  first  question  asked  students  to  rate  the  quality  of  the  VTT  instruction  for  training 
the  skills  in  the  MOS.  The  students  used  a  5-point  scale  ranging  from  excellent  to  poor  to  rate 
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the  courses.  The  second  question  asked  students  whether  or  not  they  encountered  any  problems 
completing  the  PTs.  Students  responded  either  yes  or  no.  The  final  question  asked  students 
whether  or  not  they  encountered  any  problems  passing  the  PTs.  Students  responded  either  yes 
or  no.  These  data  are  presented  in  Table  9. 

Table  9 

Student  Ratings  of  Adequacy  of  Training  for  the  MOS  Courses 


The  data  indicate  that  the  students  in  the  three  MOS  courses  thought  that  the  VTT 
instruction  was  slightly  better  than  good  (3.0)  in  teaching  the  required  skills,  as  indicted  by  their 
ratings  of  3.31  (71L10),  3.50  (76Y10),  and  3.15  (95B10).  In  addition,  at  least  60%  of  the 
students  in  each  course  responded  that  they  had  no  problems  completing  the  PTs.  When  asked 
if  they  had  any  problems  passing  the  PTs,  only  the  71L10  students  responded  that  they  had  some 
problems.  However,  student  comments  from  the  narrative  data  indicate  that  the  problems  were 
not  related  to  poor  instruction.  The  three  most  common  problems  students  gave  were:  (a) 
mistakes  in  the  PTs,  such  a  typing  errors,  (b)  time  constraints  (i.e,  there  was  not  enough  time 
allotted  to  complete  the  PTs),  and  (c)  there  was  a  mismatch  between  the  PTs  and  PEs.  These 
problems  were  beyond  the  control  of  the  FTP  because  the  PTs  and  PEs  were  obtained  from  the 
71L10  proponent  ar  !  were  used  as  directed  in  the  RC3  courseware. 
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In  summary,  both  the  quantitative  and  narrative  data  support  the  effectiveness  of  the  VTT 
instruction.  All  MOS  students  passed  all  of  the  PTs;  the  average  posttest  scores  for  the 
HazWaste  and  TQL  students  was  78%  and  93%  respectively. 


Objective  B2 

Compare  the  Training  Effectiveness  of  the  VTT  Courses  to  Standard  Military  Courses 

Taught  by  Conventional  Means 

A  staple  of  educational  media  research  is  to  compare  one  medium  to  another  medium  or 
to  compare  one  medium  to  "traditional"  instruction.  These  studies,  called  comparative 
effectiveness  studies  (CES),  typically  compare  the  relative  achievement  of  groups  of  students  who 
have  received  similar  subject  matter  from  different  media  or  from  traditional  instruction  (Clark, 
1983,  1989;  Wilkinson,  1980). 

Some  researchers  view  CES  negatively  as  there  are  too  many  differences  in  the  definitions 
of  traditional  instruction  and  media.  The  concepts  are  therefore  too  inexact  to  be  experimentally 
valid.  The  trend  in  CES  is  a  finding  of  "no  significant  differences"  between  traditional 
instruction  and  new  media. 

Rather  than  conducting  additional  comparative  effectiveness  studies,  researchers  such  as 
Clark  (1983)  and  Wilkinson  (1980)  have  suggested  comparing  the  effectiveness  of  instructional 
methods  and  the  cost  of  delivery  systems.  For  example,  the  inclusion  of  instructional  methods 
that  add  structure,  shorter  steps,  reduce  verbal  loads,  allow  self-pacing,  provide  cueing  and 
prompting,  and  repeated  exposures  to  content,  etc.,  is  often  provided  in  newer  media  and  is  the 
result  of  more  instructional  design  and  development.  Presumably,  however,  any  courseware 
(including  traditional  instruction)  could  benefit  greatly  if  effort  went  into  the  design  and 
development  of  the  course. 

Only  Go/No  Go  data  were  available  from  the  proponent  schools  and  this  would  not  allow 
a  valid  comparison  to  be  made  between  VTT  and  traditional  instruction.  Therefore,  instead  of 
comparing  the  achievement  of  students  in  traditional  versus  VTT  instruction,  a  rough  comparison 
has  been  made  between  the  instructional  methods  and  time  frames  used  in  each  VTT  course  and 
its  traditional  counterpart. 

Comparison  of  Instructional  Methods 

The  comparison  of  instructional  methods  for  the  Army  MOS  courses  (71L10, 76Y10  and 
95B10)  was  made  between  the  two  traditional  alternatives  normally  available  to  the  training 
population.  Currently,  RC  personnel  may  become  qualified  in  a  new  MOS  by  attending  a 
resident  course  at  the  proponent  school  or  attending  a  one  or  two-phased  RC3  taught  at  a  local 
USARF  school.  The  RC3  versions  typically  cover  the  same  technical  topics  as  the  proponent 
school  resident  course,  but  they  spend  less  time  on  each  topic  because  RC  personnel  have  more 
Army  experience.  Many  RC  personnel  have  already  performed  some  duties  that  will  be  required 
in  their  new  MOS. 

The  approach  taken  here  was  to  provide  a  general  conceptual  analysis  of  the  instructional 
methods  used  in  the  various  alternatives.  Several  general  instructional  areas  were  chosen  for 
comparison  in  each  course  because  it  was  impractical  to  compare  the  courses  completely. 
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In  71L10,  the  RC3  course  is  a  single  phased  course  that  may  be  taught  in  either  the  ADT 
or  IDT  mode.  The  71L10  VTT  course  is  an  adaptation  of  this  single  phased  RC3  course.  The 
76Y10  and  9SB10  courses  are  dual-phased  courses  that  require  students  to  complete  several 
weeks  of  IDT  training  before  taking  the  two  week  ADT  phase.  The  76Y10  and  95B10  VTT 
courses  are  adaptations  of  the  IDT  phase  of  the  76Y10  and  95B10  RC3  POIs  and  therefore,  they 
are  very  similar  to  them  in  instructional  topics  and  time  allocations.  Because  of  these 
similarities,  these  VTT  courses  are  being  compared  to  only  the  IDT  phases  of  the  RC3  materials. 
71L10  Administrative  Specialist 

Besides  comparing  the  instructional  methods  used  in  the  71L10  RC3  course,  the  71L10 
VTT  course  was  also  compared  to  the  methods  used  in  the  resident  AC  school.  The  information 
for  this  comparison  was  obtained  through  a  telephone  interview  with  Mr.  W.  Ripperger  at  the 
Soldier  Support  Center,  Ft  Benjamin  Harrison,  Indiana  on  September  20,  1993.  Ten  general 
instructional  areas  were  chosen  for  comparison  (Table  10). 


Table  10 
MOS  71L10 

Comparison  of  Alternatives 


Total  Course  Length* 

2  Weeks 

10  Days  (ADT) 
20  Days  (IDT) 

6  Weeks 

Computer  Assisted  Hours 

0 

0 

32 

Typing  Hours 

26 

26" 

80 

Practical  Exercise  Hours 

10 

39 

137*" 

Exam  Hours 

10 

15 

26 

Field  Training  Exercise  Hours 

0 

0 

40 

Lecture  Hours 

26 

26 

1 

Number  of  Graphics/Transparencies 

1200 

0 

30-35 

Color  Media 

Yes 

No 

No 

Total  Video  Minutes 

14.5 

0 

0 

Amount  of  time  required  to  complete  the  instruction. 

"Includes  .5  hours  of  lecture,  20.S  hours  of  practical  exercise  and  5  hours  of  exam. 
"‘Includes  80  hours  of  typing. 


A  unique  aspect  of  the  resident  proponent  school  course  is  that  it  has  a  three-to-five  day 
(40  training  hours)  field  training  exercise  (FTX)  during  which  the  students  practice  MOS  specific 
and  common  soldier  tasks  in  a  field  (i.e.,  non-c!-  oom)  environment.  The  FTX  has  been 
omitted  in  the  RC3  and  VTT  versions  of  the  cou  reduce  training  time.  This  design  also 
allows  the  course  to  be  taught  in  a  single  classrooi.  ,  iase. 
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Another  significant  difference  between  the  resident  course  and  the  VTT  and  RC3 
alternatives  is  that  the  resident  course  contains  36  hours  of  computer-assisted  instruction  (CAI) 
using  a  standard  word  processing  program.  In  fact,  every  student  at  the  AC  resident  school  uses 
a  computer  for  word  processing  throughout  the  classroom  portion  of  the  course.  In  both  the  RC3 
and  VTT  versions  of  the  course,  the  students  typically  use  electric  typewriters  to  practice 
keyboarding  skills;  this  does  not  give  them  practice  with  word  processing  programs. 

While  the  proponent  school  resident  course  uses  some  overhead  transparencies 
(approximately  three  to  five  per  task),  the  instructors  rely  heavily  on  the  use  of  the  chalkboard 
and  print  materials.  The  proponent  school  also  has  some  motion  media  available,  but  it  is  seldom 
used.  According  to  Mr.  W.  Ripperger,  the  instructors  at  the  proponent  school  tend  to  use  a  very 
personal  and  interactive  approach  to  delivering  the  instruction.  This  approach  may  include 
question  and  answer  sessions,  drills  designed  to  improve  the  student’s  ability  to  locate  specific 
information  contained  in  Army  regulations,  and  typing  drills.  The  CAI  portions  of  the  course 
also  contain  some  computer  generated  exercises  in  addition  to  instructor  led  keyboarding  drill  and 
practice. 

76Y10  Unit  Supply  Specialist 

Table  1 1  presents  seven  instructional  areas  that  were  chosen  for  comparison  of  the  76Y10 
courses.  Unlike  the  71L10  AC  resident  course,  the  76Y10  RC3  and  AC  resident  courses  do  not 
include  a  FI  X,  CAI,  or  typing  practice.  Lecture  and  demonstration  hours  were  grouped  together 
because  these  two  groups  were  listed  under  the  umbrella  of  "VTT  instruction"  in  the  VTT  POI, 
rather  than  separately  as  they  were  for  the  RC3  and  resident  courses.  The  information  for  the 
RC3  and  AC  resident  courses  was  obtained  through  a  telephone  interview  with  Mr.  G.  Lamar  at 
the  Quartermaster  Center  and  School,  Ft  Lee  Virginia,  on  September  28,  1993. 


Table  11 
MOS  76Y10 

Comparison  of  Alternatives 


i  _  •  \ '  f 
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is-.:"  Kpr.-..  ...  .  >"  ■ 
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Course  Length** 

10  Days 

28  Days 

49  Days  | 

Practical  Exercise  Hours 

18 

20.5 

129.5 

Exam  Hours 

15 

13 

17 

Lecture/Demonstration  Hours 

58 

67.5 

126 

Number  of  Graphics/Transparencies 

1180 

109 

200-300 

Color  Media 

Yes 

No 

Yes 

Total  Video  Minutes 

10 

0 

6-8 

w  phase  ^ 

“Amount  of  time  required  to  complete  the  instruction 


An  obvious  difference  between  the  VTT,  RC\  and  proponent  school  AC  resident  courses 
is  the  total  length  of  each  course.  The  proponent  school  resident  course  covers  more  material 
and  successful  completion  of  the  course  results  in  the  award  of  the  MOS.  The  VTT  course  and 
RC3  IDT  phase  cover  only  the  part  of  the  instruction  needed  to  perform  the  combat  critical  tasks 
related  to  the  MOS.  The  ADT  portion  of  the  RC3  course  was  not  included  in  the  VTT  course 
due  to  psychomotor  skills  content,  weapons  security  issues,  and  the  military  nature  of  the  content. 

The  six  to  eight  minutes  of  video  shown  in  the  proponent  school  AC  resident  course  is 
not  officially  part  of  the  POI.  It  is  instruction  that  is  required  to  be  shown  in  the  Basic 
Non-Commissioned  Officer  Course  (BNCOC)  and  Advanced  Non-commissioned  Officer  Course 
(ANCOC).  The  videos  shown  in  the  76Y10  AC  resident  course  are  for  information  and  future 
reference  only.  These  videos  were  not  shown  in  the  VTT  course  because  they  were  not  included 
in  the  RC3  materials  that  were  used  in  the  development  of  the  VTT  course.  The  videos  used  in 
the  76Y10  VTT  course  were  specially  produced  for  the  course  by  the  FTP  and  relate  directly  to 
the  topics  covered  by  the  instruction. 

95B10  Basic  Military  Police 

Ten  instructional  areas  were  compared  for  the  95B10  courses.  Tfie  proponent  school  AC 
resident  course  also  includes  a  47-hour  FTX  to  provide  tactical  training  and  to  allow  students  to 
practice  tactical  skills  in  the  field.  The  information  for  the  RC3  and  AC  resident  courses  was 
obtained  through  a  telephone  interview  with  Mr.  P.  Kelly  at  Military  Police  School,  Ft 
McClellan,  Alabama  on  August  30, 1993.  Like  76Y10,  the  most  obvious  difference  is  the  length 
of  time  needed  to  complete  the  courses  and  the  differences  in  the  amount  of  time  spent  on 
lecture/demonstration,  PEs,  and  examinations  (Table  12). 
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Table  12 
MOS95B10 

Comparison  of  Alternatives 


:  : '  >  '  »  >r?Slt 

. 

p  i  m 

gppptipsfp 

Total  Course  Length** 

10  Days 

11  Days 

70  Days 

Demonstration  Hours 

3 

3 

18.1 

Case  Study  Hours 

0 

0 

2 

Practical  Exercise  Hours 

20.9 

18.6 

180.6 

Examination  Hours 

17.75 

19 

47.6 

Field  Training  Exercise  Hours 

0 

0 

47 

Lecture  Hours 

38.42 

46.4 

98.7 

Number  of  Graphics, 

Slides/Transparencies 

880 

773 

500 

Color  Media 

Yes 

Yes 

Yes 

Total  Video  Minutes 

28 

0 

_ | 

** Amount  of  time  required  to  complete  the  instruction 


The  proponent  school  AC  resident  course  uses  a  mix  of  still  visual  media  consisting  of 
color  35mm  slides  and  overhead  transparencies.  This  combination  consists  of  approximately  250 
overhead  transparencies  and  250  35mm  slides.  These  35mm  slides  are  also  used  in  the  RC3  and 
some  were  used  in  the  VTT  materials.  For  the  VTT  course,  the  35mm  slides  were  digitized  and 
converted  to  computer  graphics  files  for  use  with  the  multimedia  computer.  Not  all  the  35mm 
slides  were  suitable  for  use  with  the  TNET  equipment.  The  slides  were  supplemented  by  other 
graphic  media. 

The  proponent  school  AC  resident  course  uses  interactive  strategies  that  are  designed  to 
increase  student  participation  in  the  class  lectures.  Instructors  are  trained  to  focus  on  students 
who  do  not  appear  to  be  involved  in  the  instruction.  In  addition,  the  proponent  school  resident 
course  contains  many  physical  demonstrations  that  require  student  participation.  The  180  hours 
of  PEs  in  the  proponent  school  AC  resident  course  include  psychomotor  activities  that  were  not 
included  in  the  RC3  materials  and  therefore  were  not  adapted  to  VTT. 

Hazardous  Waste  Handling,  Activity  Level 

The  HazWaste  course  delivered  by  the  FTP  was  an  adaptation  of  a  32-hour  course 
developed  by  the  Navy  Environmental  Engineering  Support  Activity  (NEESA).  The  information 
for  this  comparison  came  from  an  interview  with  Ms.  U.  Shaw,  Naval  Air  Station,  Cecil  Field, 
Jacksonville,  Florida  on  October  13,  1993. 
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The  NEESA  course  is  normally  used  to  train  HazWaste  coordinators  in  and  above  the 
grade  of  E7.  Students  in  the  32-hour  NEESA  course  are  evaluated  on  a  pass/fail  basis  and 
successful  completion  of  the  course  results  in  certification  as  a  HazWaste  coordinator. 

The  VTT  HazWaste  course  combined  elements  of  the  32-hour  NEESA  coordinators  course 
with  safety  considerations.  It  was  designed  as  an  overview  for  all  HazWaste  handlers  and 
resulted  in  no  official  military  certification.  An  eight  hour  platform  delivered  course,  similar  to 
the  VTT  course,  has  been  planned  by  the  Navy  but  has  not  been  developed. 

Seven  instructional  areas  were  chosen  for  comparison.  Besides  the  21  hour  difference  in 
total  course  length,  the  major  differences  were  the  number  of  hours  for  exams  and  practical 
exercises  (Table  13). 


Table  13 
HAZWASTE 

Comparison  of  Alternatives 


Course  Length 

8  Hours 

32  Hours 

Practical  Exercise  Hours 

.38 

10 

Exam  Hours* 

0 

10 

Lecture/Demonstration  Hours 

8.5  Hours 

10.5-10.75 

Number  of  Graphics/Transparencies 

150-200 

205 

Color  Media 

Yes 

Yes 

Total  Video  Minutes 

15-20 

75-90 

The  NEESA  course  instructors  use  group  participation  exercises  that  relate  the  course 
content  to  practical,  on-the-job-experiences  to  stimulate  interaction  among  the  students.  The 
instructors  also  use  volunteer  students  to  demonstrate  the  wearing  of  protective  clothing.  Guest 
demonstrators  from  the  fire  department  are  used  to  demonstrate  "encapsulated  clothing"  (clothing 
designed  to  more  completely  protect  the  wearer  from  extremely  hazardous  substances). 
Encapsulated  clothing  was  demonstrated  in  the  VTT  course  by  use  of  a  videotape  presented  over 
the  TOET  system. 


Total  Quality  Leadership 


The  VTT  TQL  course  was  an  adaptation  of  the  Navy’s  one-day  Introduction  to  TQL 
course.  Six  instructional  areas  were  chosen  for  comparison.  The  information  for  this  comparison 
was  obtained  from  Master  Chief  R.  Kline  at  Little  Creek  Amphibious  Base,  Norfolk,  Virginia  on 
October  5,  1993,  and  is  shown  in  Table  14. 
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Table  14 
TQL 

Comparison  of  Alternatives 


||  Total  Hours  of  Instruction 

6 

7 

Simulation  Exercise  Hours 

1.75 

1.5 

Total  Number  of  Simulation  Exercises 

1 

2 

Lecture  Hours 

3.75 

4.25 

Number  of  Graphics/Transparencies 

108 

50-100 

Color  Media 

Yes 

No’ 

An  interesting  difference  between  the  two  courses  is  the  number  of  simulation  exercises 
used  in  each  course.  The  standard  Navy  course  contains  two  exercises,  "Life  in  the  Red  Bead 
Factory"  and  "Broken  Squares."  The  VTT  course  only  contained  the  "Life  in  the  Red  Bead 
Factory."  Both  exercises  emphasize  team  building  and  the  importance  of  group  effort.  The 
"Broken  Squares"  exercise  also  illustrates  the  uses  of  non-verbal  communication  and  is  followed 
by  an  AAR  where  the  students  discuss  lessons  learned  from  the  exercise. 


Although  the  VTT  course  had  one  less  simulation  exercise,  it  devoted  15  minutes  more 
to  this  one  activity  than  the  standard  course  devoted  to  two  activities.  The  "Broken  Squares" 
exercise  was  omitted  from  the  VTT  course  to  compensate  for  the  extra  time  added  to  the 
instruction  by  the  network  interaction  protocols. 

A  side  note  to  the  "Life  in  the  Red  Bead  Factory"  exercise  was  the  role  of  the  MIA  in 
the  VTT  version  of  the  course  versus  the  role  of  the  military  instructor  in  the  standard  Navy 
course.  In  the  standard  course,  the  military  instructor  is  physically  present  with  the  students  and 
acts  as  a  foreman  in  the  factory.  In  the  VTT  version,  the  MIA  is  at  the  origination  site  and  acts 
as  the  corporate  Chief  Executive  Officer  (CEO).  The  role  of  the  factory  foreman  must  be  played 
by  the  IC  at  the  site.  This  required  additional  training  in  order  for  the  IC  to  adequately  perform 
this  function. 


To  summarize  this  objective,  it  was  impossible  to  compare  the  training  effectiveness  of 
different  versions  of  the  same  course.  Presumably  all  versions  are  effective,  but  the  courses 
themselves  and  the  instructional  methods  were  too  different  to  accurately  compare.  A  brief 
comparison  of  the  different  training  options  of  the  five  courses  was  provided.  Typically,  the 
VTT  courses  used  more  varied  media  and  were  usually  shorter  in  length. 
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Objective  B3 

Identify  Effective  Instructional  Methods 
and  Strategies  for  VTT  Courses 


The  purpose  of  this  objective  was  to  ascertain  which  of  the  instructional  methods  and 
strategies  used  during  VTT  were  effective.  A  review  of  literature  on  distance  learning  and 
teletraining  was  conducted  to  determine  which  instructional  methods  and  strategies  were 
suggested  for  use  during  VTT  (Martin,  1993).  The  methods  and  strategies  that  were  most 
important  for  incorporation  into  VTT  courseware  were: 

•  involvement  and  motivational  activities 

•  interaction  with  the  instructor,  other  students,  and  the  instructional  materials 

•  feedback 

•  advance  organizers 

•  group  and  individual  activities 

•  opportunities  for  students  to  ask  questions 

•  a  student  study  guide 

•  off-line  activities 

•  a  multimedia  approach,  including  the  use  of  graphics,  other  visuals,  and  video. 

During  the  design,  development,  and  production  phases  of  the  FTP,  the  course  developers 
used  principles  of  instructional  systems  design  (ISD),  specifically  the  conditions  of  learning  and 
the  events  of  instruction  (Gagne,  1985;  Gagne,  Briggs,  &  Wager,  1989).  The  suggested  VTT 
methods  and  strategies  were  integrated  with  the  principles  of  ISD  and  incorporated  into  the 
instruction.  Specific  examples  of  how  these  methods  were  included  in  the  VTT  courses  are: 

•  Instructional  games  were  developed  to  promote  involvement,  motivation,  and 
interaction. 

•  PEs  were  used  to  provide  practice,  feedback,  and  remediation,  and  they  were 
designed  as  individual  and  group  on-line  and  off-line  activities. 

•  Question  and  answer  sessions  were  preplanned  and  included  in  the  instruction  to 
provide  practice,  feedback,  interaction,  and  remediation. 
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Advance  organizers  were  included  to  provide  an  overview  and  context  for:  (a)  the 
course  (in  part  by  presenting  the  goals  and  objectives  of  the  course),  (b)  for  each 
lesson,  and  (c)  for  each  videotape  presentation. 

A  student  study  guide,  called  the  Interactive  Study  Guide  (ISG),  was  designed  to 
promote  interaction  with  the  instructional  materials  and  to  focus  the  student’s 
attention  on  important  ideas  and  concepts  that  should  be  learned  and  remembered. 

Word  pictures  (Cyrs  &  Smith,  1990)  were  incorporated  into  each  ISG.  Word 
pictures  are  graphic  representations  of  the  content  with  key  words  omitted  (Figure 
14).  Students  wrote  in  the  important  points  in  the  ISG  as  the  instruction 
progressed. 


*  student  fills  in  word  as  shown  on  the  screen  graphics 


Figure  14.  Example  of  a  Word  Picture 
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•  Graphics  were  used  extensively  in  all  the  courses  to  support  the  instruction,  focus 
attention,  and  provide  motivation. 

•  Multimedia,  including  videotapes  and  the  use  of  the  ELMO,  was  used  as  needed 
to  explain  important  concepts,  ideas,  and  procedures,  to  clarify  content,  to  provide 
practice  and  feedback  activities,  and  to  stimulate  attention  and  motivation. 

Data  from  Students 

Students  were  asked  to  rate  on  a  5-point  scale:  (a)  the  effectiveness  of  instructional 
methods  that  were  used  in  the  courses,  (b)  the  amount  of  interactivity  provided,  and  (c)  other 
course  characteristics  that  would  provide  input  into  the  effectiveness  of  the  VTT  courses. 

Instructional  Methods.  Figure  15  and  Table  B3.1  (in  Appendix  F)  present  the  student 
ratings  of  instructional  methods.  Students  in  the  three  MOS  courses  rated  the  word  pictures  and 
graphics  between  moderately  helpful  and  helpful  (3.65,  71L10;  3.82,  76Y10;  3.44,  95B10) 
whereas  students  in  the  special  topics  courses  rated  the  graphics  and  word  pictures  between 
helpful  and  very  helpful  (4.33,  HazWaste;  4.43,  TQL).  In  the  MOS  classes,  over  75%  of  the 
students  rated  word  pictures  and  graphics  3.0  or  higher  ( moderately  helpful  or  better);  in  the 
special  topics  classes,  over  95%  of  the  students  rated  the  use  of  word  pictures  and  graphics 
moderately  helpful  or  better. 

With  the  exception  of  76Y10,  all  students  rated  the  value  of  the  PEs  as  helpful  (over  4.0). 
While  the  76Y10  students  ranked  the  PEs  as  moderately  helpful  (3.44),  over  90%  of  the  students 
in  all  of  the  classes  including  76Y10  rated  the  PEs  as  3.0  or  above.  Based  on  their  narrative 
responses,  some  of  the  comments  students  made  about  the  76Y10  proponent  school  supplied 
practical  exercises  were  that  they  were  difficult  to  read  because  of  poor  reproduction  quality,  they 
contained  both  content  and  typographical  errors,  and  they  were  too  long. 

Students  in  the  MOS  courses  were  also  asked  about  the  effectiveness  of  the  instructional 
games  and  the  remediation  activities  used  in  the  courses.  The  instructional  games  were  rated 
between  moderately  helpful  and  helpful  (3.44,  76Y10;  3.46,  95B10),  with  the  71L10  students 
rating  games  lower  (2.61)  than  the  other  two  classes.  Remediation  activities  were  also  rated 
helpful  by  the  71L10  and  95B10  students  (4.13, 4.05),  with  76Y10  students  rating  these  activities 
lower  (3.68)  than  the  other  two  groups. 

Special  instructional  methods  were  used  in  each  of  the  MOS  classes  as  required  by  the 
POIs.  These  are  listed  in  Table  B3.1  and  were  rated  helpful  (4.39,  71L10;  3.82,  76Y10;  4.60, 
95B10)  by  the  students. 

Interactivity.  One  of  the  key  principles  required  for  effective  VTT  instruction  is  that 
students  must  have  opportunities  to  ask  questions  and  to  interact  with  each  other  and  instructional 
personnel  at  the  origination  and  their  site. 
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Figure  15.  Student  Ratings  of  Instructional  Methods 


Students  were  asked  to  rate  seven  different  types  of  interactivity: 

•  opportunities  to  ask  questions 

•  opportunities  to  ask  follow-up  questions 

•  interaction  with  students  on-site 

•  interaction  with  students  on  the  network 

•  interaction  with  the  VTT  instructor 

•  interaction  with  the  instructional  coordinator  (IC) 

•  interaction  with  the  military  site  coordinator  (MSC) 

Table  B3.2  in  Appendix  F  gives  the  means,  standard  deviations,  percentage  of  students 
rating  the  interactivity  of  the  course  as  3.0  or  higher  on  a  5-point  scale,  and  the  sample  size  for 
each  of  the  five  courses.  Figure  16  graphically  presents  the  means  for  representative  types  of 
interactivity. 

Of  the  29  categories  regarding  interactivity,  24  were  rated  at  4.0  or  above,  indicating  that 
the  students  felt  the  opportunities  to  ask  questions  and  to  interact  with  others  during  instruction 
was  very  good  or  helpful.  Of  the  five  rating  below  4.0,  three  of  them  were  on  the  question 
interaction  with  students  on  the  network.  Students  in  the  three  MOS  courses  rated  this  type  of 
interactivity  as  3.38  (71L10),  3.50  (76Y10),  and  3.08  (95B10j.  Students  in  the  special  topics 
courses  were  not  asked  this  question.  A  review  of  students’  responses  from  the  narrative  data 
from  the  three  MOS  courses  revealed  that  the  students  at  the  FCCJ2  site  felt  more  isolated  than 
did  the  students  at  the  other  sites.  These  data  also  revealed  that  the  students  at  SPJC  and  VCC 
had  more  negative  perceptions  about  the  students  at  FCCJ2  than  they  did  of  other  students  taking 
the  courses.  These  perceptions  may  have  influenced  the  students’  rating  of  the  network 
interactions,  but  it  is  unclear  why  the  students  at  FCCJ2  felt  more  isolated  and  why  the  other 
sites  perceived  the  FCCJ2  students  more  negatively. 

There  were  two  other  ratings  of  interactivity  that  were  below  4.0.  The  95B10  students 
rated  interaction  with  the  VTT  instructor  at  3.60;  the  76Y10  students  rated  interaction  with  the 
instructional  coordinator  at  3.92. 

Taken  as  a  whole,  these  data  about  interactivity  indicate  that  the  students  felt  that  the 
opportunities  they  had  for  asking  questions  and  interacting  with  others  during  the  VTT  instruction 
were  sufficient  in  number  and  that  they  were  instructionally  useful. 

Other  Course  Characteristics.  Students  were  also  asked  to  rate  the  quality  and 
effectiveness  of  other  aspects  of  the  courses:  the  lesson  presentations,  course  organization. 
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e  16.  Student  Ratings  of  Interactivity 


printed  materials,  time  allotted  to  cover  the  topics,  the  pacing  of  the  course,  the  introduction  to 
VTT  and  TNET,  and  the  overall  quality  of  the  course.  Table  B3.3  in  Appendix  F  gives  the 
means,  standard  deviations,  percentage  of  students  rating  the  characteristics  of  the  course  as  3.0 
or  higher  on  a  5-point  scale,  and  the  sample  size  for  each  of  the  five  courses.  Figure  17 
graphically  presents  the  means  of  representative  course  characteristics. 

In  general,  the  characteristics  Introduction  to  VTT  and  Introduction  to  TNET  equipment 
were  rated  slightly  higher  than  the  other  characteristics  with  all  the  scores  around  4.0  except  for 
76Y10.  The  76Y10  students  rated  Introduction  to  TNET  equipment  somewhat  lower  at  3.55. 

Ratings  for  the  other  course  characteristics  for  HazWaste  and  TQL  are  generally  higher 
than  for  the  MOS  courses  except  for  the  characteristic  Time  to  cover  the  topics.  While  the 
ratings  for  Time  to  cover  the  topics  were  slightly  above  average  for  the  special  topics  courses 
(3.24  and  3.13,  respectively),  it  is  possible  that  the  students  felt  that  there  was  a  great  deal  of 
content  to  be  covered  in  a  short  amount  of  time  since  these  courses  were  presented  in  one  day. 
Students  in  71L10  rated  Time  to  cover  the  topics  at  2.69,  indicating  that  they  too  thought  the 
course  may  not  have  been  long  enough  to  adequately  cover  the  content. 

When  asked  to  rate  other  characteristics  of  the  course:  quality  of  lesson  presentations, 
general  course  organization,  quality  of  printed  materials,  and  overall  quality  of  the  courses,  the 
HazWaste  and  TQL  students  tended  to  rate  those  characteristics  higher  than  did  the  MOS 
students.  The  student  ratings  from  the  MOS  courses  ranged  from  3.19  to  3.88;  for  the  special 
topics  courses,  the  ratings  ranged  from  3.84  to  4.19. 

Students  were  asked  about  the  pacing  of  the  course  they  were  taking.  A  response  of  3.0 
meant  that  the  pacing  was  about  right;  higher  numbers  meant  the  class  was  too  fast,  lower 
numbers  meant  the  class  was  too  slow.  Students  in  95B10  felt  that  the  instruction  was  a  little 
too  slow  (2.04);  students  in  71L10  felt  that  the  course  was  a  little  too  fast  (3.48).  Students  in 
76Y10,  HazWaste,  and  TQL  rated  the  pacing  of  the  course  as  2.68,  2.94,  3.1 1  respectively,  or 
about  right. 

Students  were  also  asked  whether  or  not  the  goals  and  objectives  of  the  course  were  made 
clear  to  them.  Eighty-five  percent  of  the  students  in  76Y10  said  that  the  goals  and  objectives 
were  made  clear  to  them  as  did  93.9%  of  71L10  students,  96.2%  of  95B10  students,  99.1%  of 
HazWaste  students,  and  100%  of  the  TQL  students.  These  data  indicate  that  the  students  were 
provided  an  adequate  advance  organizer  to  the  course  by  explanation  of  the  course  goals  and 
objectives. 

Data  from  Instructional  Coordinators 

The  ICs  for  the  three  MOS  courses  were  asked  to  rate  the  quality  of  the  instructional 
methods,  interactivity,  and  other  course  characteristics  at  the  end  of  each  instructional  day.  The 
ICs  rated  five  aspects  of  the  course  (Table  B3.4  in  the  Appendix  F).  These  data  are  higher  than 
the  students’  ratings.  All  ratings  except  three  are  4.0  or  higher,  with  the  71L10  course  ratings 
the  highest,  ranging  from  4.67  to  4.87. 


64 


e  17.  Student  Ratings  of  Course  Characteristics 


The  two  lowest  ratings  were  given  for  the  VTT  instructor’s  wise  use  of  available  time  for 
76Y10  and  95B10.  The  ratings  were  3.93  and  3.69  respectively.  The  only  other  rating  below 
4.0  was  3.98  ( Instructor  Use  of  Graphics  and  Charts  -  76Y10). 

These  ratings  from  the  ICs  indicate  that  they  thought  the  MOS  courses  were  of  high 

quality. 

Data  from  VTT  Instructors  and  MIAs 

The  VTT  instructors  and  the  MIAs  were  interviewed  at  the  end  of  the  course  they  were 
teaching.  Each  VTT  instructor  and  MIA  was  asked  to  discuss:  (a)  successful  and  unsuccessful 
instructional  activities  and  (b)  their  attempt  to  promote  interactivity  and  whether  or  not  they  felt 
the  amount  of  interaction  was  sufficient.  There  were  a  total  of  eleven  responses. 

In  response  to  which  strategies  were  particularly  successful,  the  VTT  instructors’  and 
MIAs’  responses  fell  into  two  main  categories:  (a)  the  multimedia  possibilities,  particularly  the 
graphics  and  word  pictures,  but  also  the  video  vignettes,  photographs,  slides,  and  the  use  of  the 
ELMO,  and  (b)  the  supplementary  exercises  (e.g.,  PEs,  case  studies,  and  question  and  answer 
sessions)  that  "broke  up  the  instruction"  and  ”provide[d]  time  for  hands-on  practice."  When 
asked  which  strategies  were  not  successful,  the  most  common  response  was  that  all  the  strategies 
"were  successful  to  some  extent." 

The  VTT  instructors  and  MIAs  were  also  asked  to  explain  how  interactivity  was  promoted 
during  the  VTT  instruction.  In  all  the  cases,  the  instructors  stated  that  the  question  and  answer 
sessions  that  were  preplanned  and  the  activities  that  promoted  discussion  (e.g.,  instructional 
games,  contests,  and  short  exercises,  including  PEs)  were  the  most  beneficial  in  stimulating 
interaction.  When  the  instructors  were  asked  to  discuss  whether  the  amount  of  interaction  in  the 
course  was  sufficient,  they  all  said  that  it  was  sufficient,  but  that  it  could  always  be  improved. 
The  most  common  suggestions  for  improvement  were  to:  (a)  make  the  interactive  activities  and 
strategies  more  novel  and  creative,  and  (b)  to  increase  the  opportunities  for  students  to  ask 
questions. 

In  summary,  the  instructional  methods  and  activities  designed  into  the  VTT  courses  were 
rated  by  students  and  instructional  personnel  as  effective.  While  improvement  is  always  possible, 
the  activities  designed  to  promote  interaction  were  the  most  highly  rated.  Since  these  activities 
were  preplanned,  this  supports  the  contention  that  well  designed  courses  influence  student 
learning. 
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Objective  B4 

Determine  the  Degree  of  Student  Motivation  and  Interest 
in  the  VTT  Instruction 


■ 


No  specific  motivational  scales  or  instruments  were  administered  to  students  to  determine 
their  degree  of  motivation  for  the  course.  However,  student  motivation  and  interest  can  be 
inferred  from  data  collected  from  the  students,  instructors,  ICs,  and  MIAs. 

Data  from  Students 

As  discussed  previously,  research  has  determined  that  interaction  is  a  critical  part  of  a 
distance  education  course.  To  increase  student  motivation  and  learning,  as  much  interaction  as 
possible  should  be  provided  between  the  instructor  and  students,  among  the  students,  and  between 
the  students  and  instructional  materials  (Ritchie  &  Newby,  1989;  Stoffel,  1987).  Objective  B3 
indicated  that  the  majority  of  students  rated  the  interactivity  in  the  VTT  courses  as  4.0  or  better 
on  a  S.O  scale.  Thus,  this  level  of  interaction  should  lead  to  high  levels  of  motivation. 

The  students’  perceptions  of  the  quality  of  the  instruction  is  also  an  indicator  of  student 
motivation.  Students  were  asked  what  they  liked  most  and  liked  least  about  this  form  of 
instruction.  Upon  analyzing  tin,  narrative  data,  evaluators  identified  several  categories  that 
encompassed  the  student  responses.  For  example,  students  commented  about  the  instructor,  the 
community  college  atmosphere,  the  technology,  course  material,  travel,  or  some  other  topics. 
Two  hundred  and  two  respondents  offered  222  positive  statements  regarding  the  courses,  while 
165  students  made  138  negative  statements.  The  large  number  of  positive  responses  indicate  that 
students  viewed  VTT  courses  favorably.  Results  are  reported  in  Table  15. 


Table  15 

Student  Ratings:  Positive  and  Negative  Statements  Regarding  VTT  Courses 


§ SOS 

MEraBi 

{~«j 

No. 

Positive 
State¬ 
ments  (N) 

49  (31) 

37  (36) 

22  (22) 

69  (68) 

45  (45) 

222  (202) 

No. 

Negative 
State¬ 
ments  (N) 

32  (30) 

32  (35) 

20  (21) 

49  (42) 

5(37) 

138  (165) 

Students  were  also  asked  to  compare  this  form  of  instruction  to  other  types  of  military 
training  using  a  5-point  scale.  Sixty-one  percent  of  the  respondents  rated  the  VTT  training  better 
than  or  superior  to  other  military  instruction. 
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In  addition,  students  were  asked  to  pick  from  a  list  of  military  training  options  the  type 
of  training  they  would  prefer  in  the  future.  Results  indicate  that  71%  of  the  respondents  ranked 
VTT  at  a  community  college  the  most  or  second-most  preferred  form  of  military  instruction.  As 
reported  in  Objective  A2,  and  illustrated  in  Figure  7,  instruction  via  V7T  at  a  community  college 
had  the  highest  mean  of  any  other  method  of  instruction.  This  finding  indicates  that  students 
were  pleased  with  this  method  of  instruction,  which  is  a  favorable  indicator  of  interest  and 
motivation. 

Finally,  students  were  asked  to  rate  on  a  5-point  scale  their  attitudes  toward  taking  another 
military  teletraining  course.  Results  were  presented  in  Figure  11  in  Objective  A2.  Student 
ratings  ranged  from  a  mean  of  3.78  in  the  71L10  course  to  4.54  in  the  TQL  course.  In  general, 
64%  expressed  a  very  favorable  response  while  15%  had  a  favorable  response. 

In  summary,  while  no  motivational  data  was  formally  collected,  the  degree  of  student 
motivation  and  interest  can  be  inferred  from  data  collected  regarding  student  perceptions  of  the 
instruction  and  its  interactivity,  their  comparisons  of  VTT  to  other  options,  and  whether  they 
would  consider  taking  another  VTT  course.  Students  were  positive  with  regard  to  each  of  those 
factors,  and  thus,  it  appears  that  they  were  very  interested  and  motivated  in  using  VTT. 
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Objective  B5 

Determine  the  Relationships  Among  Student  Entry  Characteristics  and  Student 
Performance  and  Perceptions  Regarding  VTT 


The  purpose  of  this  objective  was  to  determine  whether  student  entry  characteristics  were 
related  to  VTT  student  performance  and  course  perceptions.  It  is  useful  to  know  whether  scores 
on  the  course  PTs  are  correlated  with  student  characteristics  such  as  age,  rank,  education, 
aptitude,  prior  subject  area  knowledge,  and  so  forth.  If  they  are,  then  it  may  be  that  VTT  is  not 
for  everyone,  or  that  special  provisions  are  required  for  certain  types  of  students.  It  is  also  useful 
to  know  whether  student  ratings  of  the  VTT  approach  differ  based  upon  these  same  background 
variables.  In  this  case,  a  useful  question  is  whether  certain  types  of  students  like  the  VTT 
approach  better  than  others.  More  importantly,  do  students  perform  significantly  better  or  worse 
than  students  in  traditionally  taught  courses  and  can  these  students  be  determined  by  these 
background  variables? 

The  data  gathered  during  the  course  evaluations  offer  a  number  of  opportunities  to  explore 
these  relationships.  The  number  of  variables  available  for  the  analyses  differ  between  the  MOS 
courses  (71L10,  76Y10,  and  95B10)  and  the  two  special  topics  courses  (HazWaste  and  TQL)  in 
that  more  detailed  data  are  available  for  the  former.  However,  fewer  students  took  the  MOS 
courses,  so  less  precision  is  afforded  for  the  statistical  analyses  of  the  MOS  course  data. 

In  addressing  this  objective,  the  following  relationships  were  explored: 

•  Student  background  characteristics  to  objective  and  subjective  measures  of  course 
proficiency 

•  Student  background  characteristics  to  ratings  of  course  materials  and  processes 

•  Student  background  characteristics  to  overall  course  ratings. 

Variables  Used  in  the  Analysis 
Student  Background  Variables 

A  number  of  student  background  variables  were  selected  for  this  analysis.  Most  of  the 
background  variables  were  available  from  students  in  all  of  the  MOS  courses  and  many  were 
available  from  the  HazWaste  and  TQL  students.  The  categories  of  background  variables  that 
were  analyzed  are  described  below  (variable  label  in  parentheses): 

•  Date  of  birth  (DOB) 

•  Grade/Highest  grade  reached  (Grade) 

•  Primary  Military  Occupational  Specialty  and  duty  position  (PMOS) 
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•  Expiration  Term  of  Service  date  and  reenlistment  intention  (ETS) 

•  Level  of  civilian  education  (Education) 

•  Reason  for  taking  course,  interest  in  course,  and  relevancy  to  military  goals 
(Interest) 

•  Previous  experience  with  distance  courses  and  technology  (DE  Experience) 

•  Leisure  activities  -  reading,  television  (Leisure) 

•  Number  of  related  courses  taken  -  HazWaste  and  TQL  (Courses) 

•  Amount  of  related  prior  experience  (Experience) 

•  Armed  Services  Vocational  Aptitude  Battery  scores  -  MOS  courses  only  (ASVAB) 

Objective  and  Subjective  Measures  of  Course  Performance 

Measures  of  student  performance  varied  by  course  as  described  in  Objective  Bl.  Pretests 
and  posttests  for  the  MOS  courses  were  taken  directly  from  the  course  materials,  and  FFP  t  'jff 
constructed  objective  tests  for  HazWaste  and  TQL.  Pre  and  posttesting  were  implemented  for 
HazWaste,  but  not  for  TQL.  Thus,  objective  posttest  scores  were  available  for  all  of  the  courses, 
but  gain  scores  (difference  between  pretest  and  posttest  scores)  could  not  be  calculated  for  TQL. 
Subjective  measures  (self-ratings)  of  pre  and  post  course  proficiency  were  available  only  for  the 
three  MOS  courses. 

Ratings  of  Course  Materials  and  Processes 

A  number  of  items  from  questionnaires  administered  in  each  of  the  courses  were  selected 
as  measures  of  student  course  perceptions.  These  items  consisted  of  student  ratings  of  course 
materials  and  processes.  Each  question  required  the  students  to  respond  on  a  5-point  Likert  scale. 
Responses  to  these  items  were  summed  to  obtain  a  composite  rating  of  the  perceived  quality  of 
the  materials  and  processes  used  in  the  courses. 

Overall  Course  Rating 

Eight  items  measured  the  students’  overall  ratings  of  the  VTT  courses.  These  were  also 
measured  on  5-point  Likert  scales.  Again,  responses  were  summed  to  obtain  an  overall  course 
rating  for  each  student. 


Methodology 

The  statistical  analyses  varied  somewhat  from  course  to  course  because  of  the  differing 
data  sets  available.  However,  the  general  approach  for  the  analyses  was  two-fold.  First,  zero- 
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order  Pearson  product-moment  correlations  were  calculated  between  each  of  the  student 
background  variables  and  the  corresponding  outcome  measures.  This  was  done  to  gain  a  general 
understanding  of  the  types  of  relationships  between  the  predictor  and  criterion  variables. 

Second,  stepwise  multiple-regression  analyses  were  performed  on  the  variables  within 
each  class  of  performance  measures  and  ratings  to  determine  the  best  prediction  equation  for  each 
case.  Variables  were  entered  into  the  prediction  equations  based  upon  their  predictive  qualities 
and  eliminated  only  when  the  associated  significance  tests  for  exit  failed  to  reach  the  standard 
.05  significance  level. 


Results 

Relationship  of  student  background  variables  and  objective  and  subjective  measures  of 
course  performance 

The  question  addressed  here  was  whether  the  students’  performance  in  the  courses  was 
related  to  the  background  and  skills  that  they  brought  to  the  courses.  Pre  and  post  measures  of 
objective  proficiency  were  available  for  all  courses  except  for  TQL,  where  students  only  took  the 
posttest.  Subjective  pre-post  self-assessments  of  proficiency  were  available  only  for  the  MOS 
courses. 

Correlations.  Pearson  product-moment  correlation  coefficients  were  calculated  to 
determine  the  relationship  of  each  student  background  variable  with  proficiency  on  the 
performance  tests  (i.e.,  posttest  and  gain  scores).  To  account  for  the  fact  that  there  was  a  small, 
but  existing  level  of  missing  data  in  each  of  the  analyses,  pairwise  deletion  of  cases  was  used 
when  calculating  the  correlation  coefficients  and  testing  their  significance. 

Twenty  demographic  variables  were  correlated  with  14  measures  of  subjective  and 
objective  performance.  That  is,  for  each  course  except  71L10,  the  20  demographic  variables 
were  correlated  with  one  subjective  rating  of  post-course  achievement  (MOS  courses  only)  and 
two  performance  scores  (posttest  and  gain  scores,  except  for  TQL  which  only  had  a  posttest). 
For  71L10,  there  were  four  performance  measures,  a  posttest  and  gain  score  for  both  typing  skill 
and  memorandum  writing.  The  results  of  the  correlational  analysis  are  provided  in  Table  B5.1 
in  Appendix  G. 

In  general,  demographic  variables  did  not  correlate  with  subjective  or  objective  measures 
of  performance.  Of  the  280  correlations  calculated,  only  42  (15%)  were  significant.  Considering 
the  number  calculated,  one  may  wonder  if  the  significant  correlations  are  real  or  due  to  chance 
factors.  One  may  employ  the  Bonferroni  multi-stage  correction  to  control  for  potentially  spurious 
findings  (Larzel ere  &  Mulaik,  1977).  Using  this  approach,  the  accepted  level  of  significance 
(a=.05)  is  divided  by  the  number  of  tests  conducted  (280),  resulting  in  a  new  significance  level 
equal  to  .0002.  The  most  significant  correlation  was  p=.001;  thus,  none  of  the  correlations  are 
significant  after  correction.  However,  the  reader  should  note  that  when  large  numbers  of  analyses 
arc  employed,  the  Bonferroni  procedure  is  highly  conservative. 
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Examining  the  correlations  that  were  found  prior  to  the  Bonferroni  correction,  the  general 
finding  is  that  the  variables  positively  related  to  perceived  achievement  and/or  to  higher 
proficiency  test  scores  in  the  MOS  courses  were:  (a)  related  skills  (e.g.,  computer  skills),  (b) 
having  attended  the  associated  military  training  school,  and  (c)  several  ASVAB  scores.  A 
positive  correlation  between  demographic  variables  related  to  knowledge  and  experience  in  the 
course  content  and  subjective  ratings  of  post-course  achievement  and  actual  performance  would 
be  expected  since  related  experience  should  yield  improved  performance. 

However,  some  negative  correlations  were  found  (Table  B5.1  in  Appendix  G).  For 
example,  current  grade  was  found  to  be  negatively  related  to  both  the  objective  and  subjective 
typing  posttests  in  the  71L10  course.  Thus,  students  of  lower  rank  performed  better  than  those 
of  higher  rank,  and  similarly,  self-assessments  of  post-course  achievement  for  students  of  lower 
rank  were  higher  than  those  of  students  possessing  higher  rank.  As  to  why  lower  ranking 
students  performed  better,  it  is  possible  that  they  had  more  familiarity  with  the  required  tasks 
than  higher  ranking  students.  The  latter  may  have  been  removed  from  these  types  of  tasks  for 
a  longer  duration. 

Similarly,  negative  correlations  were  found  in  the  76Y10  course  (Table  B5.1  in  Appendix 
G).  Length  of  time  in  current  PMOS  and  Relevance  of  the  current  PMOS  were  negatively 
correlated  with  both  self-assessment  of  post-course  achievement  and  with  actual  performance. 
Relevance  of  the  current  duty  position  was  also  negatively  correlated  with  the  self-assessment 
of  post-course  achievement.  The  reason  for  these  negative  correlations  may  be  due  to  the  fact 
that  students  were  being  taught  procedures  applicable  to  combat  operations,  but  not  the 
procedures  required  for  their  present,  peace-time  MOS.  That  is,  the  students  complained  that 
they  were  learning  different  procedures  in  class  than  they  were  trained  ?  rform  in  their  daily 
jobs.  Thus,  the  relevance  of,  and  length  of  time  in,  the  PMOS  or  ck  position  may  have 
adversely  affected  perceived  and  actual  performance  in  the  course. 

Multiple  Regression  Analyses.  Results  of  the  stepwise  multiple  regression  analyses  for 
predicting  scores  on  the  proficiency  tests  from  student  background  variables  are  included  in  Table 
B5.2  located  in  Appendix  G.  In  these  and  the  other  regression  analyses  reported  herein,  where 
missing  data  problems  were  encountered,  pairwise  deletion  of  cases  was  used. 

While  a  number  of  zero-order  correlations  between  student  background  characteristics  and 
the  performance  were  found  (Table  B5.1),  the  size  of  the  correlations  was  generally  modest. 
Thus,  the  regression  equations  contained  relatively  few  variables.  The  obtained  multiple  R2s 
(proportions  of  criterion  variance  explained)  were  between  .13  and  .44.  Prior  training  at  the 
relevant  military  school,  related  skills  (e.g.,  computer  experience),  and  ASVAB  Skilled  Technical 
(ST)  score  were  the  primary  predictors  of  objective  posttest  success  in  the  MOS  courses. 
Proficiency  in  the  HazWaste  course  was  best  predicted  by  holding  a  related  military  position,  and 
proficiency  in  TQL,  with  its  more  cerebral  content,  was  best  predicted  by  college  experience  and 
prior  related  military  training. 

Table  B5.3  in  Appendix  G  presents  a  summary  of  the  regression  results  for  the  pretest  to 
posttest  gain  scores.  The  multiple  R2s  varied  from  .08  to  .67,  the  highest  being  that  for  76Y10. 
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I  Objective  performance  in  the  MOS  courses  was  best  predicted  by  prior  experience  and  skills, 

certain  technical  skill  levels  (e.g.,  computer  skills  as  they  relate  to  typing),  aptitude  measures 
such  as  certain  ASVAB  scores,  and  the  types  of  positions  currently  held.  Gain  in  the  HazWaste 
course  was  best  predicted  by  previous  coursework  in  environmental  science  and  interest  in  the 
course.  No  pretest  data  were  available  for  the  TQL  course,  making  the  corresponding  regression 
|  analysis  impossible  in  this  case. 

Results  of  the  stepwise  multiple  regression  analyses  for  predicting  students’  perceived 
I  proficiency  and  perceived  gain  in  proficiency  are  presented  in  Tables  B5.4  and  B5.5  in  Appendix 

I  G.  The  best  predictors  of  perceived  proficiency  in  the  MOS  courses  were  factors  such  as  current 

duty  position  or  rank,  prior  college  courses,  experience  with  television  courses,  and  one  ASVAB 
score  (OF:  Operators  and  Food  Handlers).  The  calculated  multiple  R2s  ranged  from  .11  to  .71 
'  for  the  three  MOS  courses,  the  highest  being  that  for  95B10. 

I  Self-assessed,  pre-post  gain  in  one  of  the  MOS  courses  was  predicted  by  an  ASVAB  score 

1  while  position  and  previous  experience  were  predictors  for  another  (Table  B5.5).  For  the  95B 10 

course  there  were  no  significant  predictors.  The  two  calculated  R2s  were  .41  for  7 1L 10  and  .72 
I  for  76Y10. 

.  Relationship  of  student  background  variables  and  student  ratings  of  course  materials 

I  and  processes 

■  The  question  addressed  in  this  section  was  whether  students’  ratings  of  course  materials 

l  and  processes  were  related  to  the  background  and  skills  they  brought  to  the  courses.  The  answer 

appears  to  be  "no." 

I  Correlations.  With  the  exception  of  the  correlation  results  for  95B 1 0,  few  variables  were 

related  to  the  composite  rating  of  course  materials  and  processes  (see  Table  BS.6  in  Appendix 
I  G).  Of  the  105  correlations  conducted,  only  six  (6%)  were  significant.  As  before,  if  the 

I  Bonferonni  correction  is  applied  here,  none  of  these  correlations  are  significant.  Thus,  these 

results  must  be  interpreted  with  caution.  In  95B10,  current  position  and  experience  were 

I  negatively  correlated  with  the  ratings,  although  age  was  positively  related.  Generally,  it  appears 

that  less  MOS  experienced  students  appreciated  the  course  materials  and  processes  more  than 
older,  more  experienced  students.  The  novelty  of  the  courses  may  have  been  more  impressive 
|  to  the  less  experienced  students. 

The  only  other  significant  correlations  were  obtained  for  the  ratings  of  the  HazWaste 
I  course.  Interest  in  this  course  and  prior  experience  in  TV  courses  produced  small,  but  significant, 

"  positive  correlations  with  the  rating  of  materials  and  processes. 

I  Multiple  Regression  Analyses.  The  summary  of  regression  results  for  the  composite 

rating  of  course  materials  and  processes  is  presented  in  Table  B5.7  in  Appendix  G.  Few  strong 
i  predictive  relationships  were  found  among  the  background  variables  for  predicting  student  ratings. 

I  The  two  significant  predictors  were  months  in  the  current  PMOS  for  95B10  and  course  interest 

for  the  HazWaste  course.  The  R2s  were  .49  and  .23  for  95B10  and  HazWaste,  respectively. 

I 
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Relationship  of  student  background  variables  and  students’  overall  course  ratings 

The  question  to  be  answered  in  this  section  was  whether  students’  overall  course  ratings 
were  related  to  the  background  and  skills  they  brought  to  the  courses.  As  in  the  previous  section, 
the  answer  appears  to  be  "no." 

Correlations.  Again,  few  significant  correlations  were  obtained  (see  Table  B5.8  in 
Appendix  G).  Of  the  105  correlations  conducted,  only  12  (11%)  were  significant.  If  the 
Bonferonni  correction  is  used,  none  of  these  correlations  are  significant.  However,  examining 
these  results,  the  significant  correlations  were  found  mostly  in  the  MOS  courses.  Overall  course 
rating  was  positively  related  to  several  student  background  characteristics  (e.g.,  months  in  current 
PMOS,  intent  to  re-enlist).  Negative  correlations  were  observed  mainly  for  the  education 
variables  in  the  71L10  and  76Y10  courses.  Apparently,  more  education  resulted  in  less  favorable 
overall  ratings  of  these  two  courses,  perhaps  because  these  students  were  likely  to  be  performing 
more  upper  level  jobs,  and  hence,  they  did  not  appreciate  these  tasks,  they  no  longer  performed 
them,  or  both. 

Multiple  Regression  Analyses.  The  results  of  the  regression  analyses  for  predictin '  the 
overall  course  ratings  from  the  predictor  variables  are  summarized  in  Table  B5.9  in  Appendix 
G.  The  best  predictors  of  overall  course  ratings  were  months  of  college,  months  in  PMOS,  the 
perceived  utility  of  the  college  credit  awarded,  ASVAB  General  Mechanical  score,  and  age.  R2s 
varied  from  .04  to  .52  across  the  five  courses  with  those  for  76Y10  and  95B10  being  the  highest. 

In  summary,  this  objective  was  assessed  using  correlation  and  multiple  regression 
techniques  in  an  attempt  to  identify  student  demographic  characteristics  that  predict  success  in 
video  teletraining  courses.  Of  the  490  correlations  computed,  only  60  were  significant  (12%). 
Due  to  the  potential  for  spurious  significant  results  because  of  the  large  number  of  correlations, 
a  correction  of  the  standard  .05  significance  level  may  be  warranted.  For  example,  if  a 
Bonferonni  correction  is  applied,  then  none  of  these  correlations  were  significant  (note  that  this 
procedure  is  very  conservative).  Therefore,  the  results  of  both  correlation  and  multiple  regression 
analyses  indicate  that  no  particular  demographic  variables  predict  success  in  VTT  courses 
differently  than  those  that  would  be  expected  to  predict  success  for  courses  taught  traditionally 
(e.g.,  related  experience,  technical  skills).  Based  on  these  analyses,  VTT  appears  to  be 
appropriate  for  any  military  student  population. 
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Objective  B6 

Determine  the  Effectiveness  of  the  Teletraining  Instructors 
and  Site  Coordinators  (Community  College  and  Military)  in  Providing 

Successful  VTT  Instruction 

Teletraining  instructors  comprise  the  core  of  the  TNET  instructional  team,  supported  at 
the  delivery  site  by  the  MIA  and  at  the  remote  sites  by  the  IC  and  the  MSC.  Evaluating  the 
effectiveness  of  this  team  was  achieved  primarily  from  the  students’  perspectives,  with  additional 
input  from  the  other  members  of  the  team,  i.e.,  the  VTT  instructor,  the  IC  and  the  MIA.  Most 
assessments  of  the  team’s  effectiveness  were  made  by  the  student  about  the  instructor.  Student 
ratings  of  effectiveness  of  the  MIA,  IC  and  MSC  were  not  as  varied  and  numerous  as  those  of 
the  VTT  instructor. 

As  in  other  estimates  of  instructional  effectiveness,  some  allowance  should  be  made  for 
the  novelty  of  the  instructional  technology  and  course  design  that  may  have  influenced 
participants’  evaluations  of  instructional  team  members.  Students  may  have  responded  positively 
to  an  instructor  or  a  course  simply  because  they  were  impressed  or  excited  by  the  newness  of  the 
instructional  system.  In  this  evaluation  there  was  no  attempt  to  control  for  this  effect.  Further, 
conventional  classroom  instruction  does  not  include  the  support  members  required  for  VTT.  The 
novelty  of  these  team  members  may  have  affected  participants’  responses.  In  conventional 
instruction,  all  roles  would  be  consolidated  into  that  of  the  instructor. 

The  technology  could  have  functioned  as  an  obstacle  to  evaluating  the  instructor’s 
effectiveness.  Students  often  had  a  more  positive  reaction  to  the  IC  than  to  the  instructor.  This 
response  is  perhaps  not  surprising,  since  die  IC  was  in  the  classroom  with  the  student  and  was 
able  to  have  more  ongoing  personal  contact.  Indeed,  the  IC  was  expected  to  be  the  instructor’s 
representative  in  the  classroom,  providing  the  student  with  the  personal  interaction  not  available 
from  the  instructor  through  the  TNET  system.  The  two-way  audio  and  video  offered  some 
student-instructor  interaction,  but  it  could  not  replace  the  in-class  contact  found  in  a  conventional 
classroom  setting. 

Just  as  the  technology  functioned  as  the  path  connecting  the  instructor  to  the  students,  it 
also  served  as  a  barrier  between  the  two,  making  it  more  difficult  for  the  students  to  have  a 
personal  relationship  with  the  instructor.  One  way  that  the  instructor  overcame  this  barrier  was 
through  the  use  of  humor  and  other  interactive  techniques.  Narrative  accounts  suggested  that 
instructors  achieving  higher  scores  on  measures  of  effectiveness  were  those  who  appeared  to  be 
successful  in  personalizing  the  instruction  despite  the  technological  obstacles  placed  before  them. 
However,  another  possibility  was  that  those  were  better  instructors  in  general  and  the  technology 
had  little  impact  on  their  techniques.  The  evaluation  did  not  attempt  to  control  for  or  analyze 
these  variables. 

Student  Ratings  of  the  Instructor 

Students  were  asked  to  rate  a  series  of  activities  on  a  5-point  scale  in  regard  to  how  much 
the  activities  assisted  in  understanding  the  content  of  the  course.  The  results  of  the  student 
ratings  that  were  assessed  in  all  five  courses  are  provided  in  Table  16.  Means,  standard 
deviations,  sample  size,  and  the  percent  of  respondents  rating  the  activity  as  Helpful  and  Very 
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Helpful  (scale  items  4  and  5  respectively)  are  presented.  The  first  activity  measured  in  this  series 
was  the  interaction  with  the  instructor  Uh  the  strongest  responses  coming  from  the  HazWaste 
students;  89%  of  them  rated  the  interaction  with  the  instructor  either  Helpful  or  Very  Helpful  in 
understanding  the  contents  of  the  course. 

Table  16 

Student  Ratings  of  the  VTT  Instructor 


Interaction  with  Instructor  on  Network 


mm 

X  (SD) 

4.23  (.88) 

4.15  (1.08) 

3.60(1.15) 

4.33  (.74) 

4.23  (.73)  I 

N 

32 

40 

25 

111 

47 

%  Helpful/ 

Very  Helpful 

84 

70 

56 

89 

87 

instructor’s  Poise,  Personality  and  Enthusiasm 


X  (SD) 

4.36  (.78) 

4.70  (.52) 

3.81  (.90) 

4.20  (.97) 

4.60  (.64) 

N 

33 

40 

26 

111 

48  1 

|  %  Very  Good/ 

|  Excellent 

88 

98 

65 

81 

92 

itructor’s  Communication  Skills 


- 

X  (SD) 

4.30  (.81) 

4.50  (.93) 

3.96  (.82) 

4.20  (.95) 

4.46  (.87) 

N 

33 

40 

26 

111 
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Students  also  rated  the  instructors  in  regard  to  various  facets  of  their  teaching  skills.  One 
item  in  this  category  measured  the  students’  evaluations  of  the  instructor’s  poise,  personality  and 
enthusiasm.  The  responses  ranged  from  65%  of  the  students  rating  the  instructor  as  very  good 
or  excellent  in  95B10  to  98%  in  76Y10. 


The  instructor’s  communication  skills  were  rated  next.  The  76Y 10  instructor  received  the 
highest  ranking  on  this  variable;  93%  of  the  respondents  in  that  course  ranked  his  communication 
skills  Very  Good  or  Excellent.  Seventy-three  percent  of  the  respondents  in  the  95B10  class 
ranked  the  instructor  as  Very  Good  or  Excellent. 

Regarding  the  ratings  of  the  instructor’s  preparation  to  teach  on  the  VTT  network,  87% 
of  the  respondents  in  the  TQL  course  rated  the  instructor  as  Very  Good  or  Excellent ,  while  only 
66%  of  the  respondents  in  the  71L10  course  rated  the  instructor  as  Very  Good  or  Excellent  on 
this  measure. 

The  overall  assessment  of  the  instructor  from  the  students’  perspectives  was  also  rated. 
The  highest  rating  was  given  to  the  TQL  instructor,  who  received  a  rating  of  Very  Good  or 
Excellent  on  this  question  by  89%  of  her  students.  Students  in  9SB10  rated  their  instructor  with 
an  average  of  almost  4.0  (Very  Good)  and  66%  of  the  respondents  rated  him  as  Very  Good  or 
Excellent. 

A  number  of  student  ratings  were  obtained  only  in  the  MOS  courses.  The  results  of  these 
ratings  are  provided  in  Table  17.  The  students  rated  the  instructor’s  delivery  of  information  over 
the  network.  The  next  item  asked  the  students  to  rate  the  instructor  on  his  or  her  organization 
of  class  sessions.  Students  were  also  asked  to  rate  the  effort  of  the  instructor  to  encourage  class 
participation  and  to  rate  the  instructors’  ability  to  make  good  use  of  the  time  available.  With  the 
exception  of  one  rating  in  the  93B10  course,  all  ratings  were  near  4.0  indicating  that  the  students 
thought  their  instructors  were  very  good. 

Finally,  a  measure  of  instructor  effectiveness  was  the  students’  evaluations  of  the 
instructors’  effective  use  of  the  equipment.  The  students  in  76Y10  rated  their  instructor  highest 
in  this  regard,  with  76%  of  them  gauging  him  as  Very  Good  or  Excellent.  The  instructor  with 
the  lowest  rating  had  61%  of  the  student  respondents  ranking  him  Very  Good  or  Excellent  in  the 
effective  use  of  the  equipment  (Table  17). 

The  narrative  data  from  the  students  support  the  findings  presented.  However,  students 
consistently  raised  questions  about  whether  the  civilians  were  knowledgeable  enough  about 
military  content  to  present  the  MOS  courses. 
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Table  17 

Student  Ratings:  Additional  Factors  for  MOS  Courses 
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Student  Ratings  of  the  Military  Instructional  Assistant  (MIA) 

Students  also  rated  the  performance  of  the  MIA.  The  role  of  the  MIA  was  to  support  the 
civilian  VTT  instructor  at  the  delivery  site;  for  example,  he  or  she  was  to  answer  instructional 
questions  from  the  military  perspective  and  substitute  for  the  VTT  instructor  when  appropriate. 
Therefore,  results  from  this  item  should  be  viewed  carefully  since  there  was  a  minimum  of 
contact  between  students  and  the  MIA.  These  results  are  presented  in  Table  18.  At  the  top  of 
the  scale,  85%  of  the  students  in  both  the  71L10  and  TQL  courses  gave  their  MIA  a  rating  of 
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Very  Good  or  Excellent .  Only  57%  of  the  students  in  the  76Y10  course  rated  their  MIA  as  Very 
Good  or  Excellent,  though  the  mean  response  was  3.64. 


Table  18 

Student  Ratings:  Overall  Performance  of  the  MIA 
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Student  Ratings  of  the  Instructional  Coordinator  (IC) 


Students  were  also  asked  to  rate  how  helpful  their  interaction  with  the  IC  was  in 
understanding  the  content  of  the  course  (Table  19).  The  ratings  for  the  ICs  are  much  higher  than 
ratings  of  the  instructor  on  the  same  question.  Ninety-six  percent  of  the  students  in  71L10  rated 
interaction  with  the  IC  Helpful  or  Very  Helpful  in  understanding  the  content  of  the  course.  The 
lowest  rating  was  in  the  76Y10  course,  where  only  73%  of  the  respondents  judged  the  interaction 
with  the  IC  to  be  Helpful  or  Very  Helpful  in  understanding  the  content  of  the  course.  All  but  one 
of  the  means  on  this  item  were  over  4.0  (Helpful).  Overall,  students  appeared  to  consider  the 
IC  an  important  part  of  the  program.  As  noted  earlier  in  this  objective,  the  narrative  data 
indicated  that  the  students'  daily  personal  contact  with  the  IC  led  to  a  more  positive  evaluation 
of  the  IC  when  compared  to  the  instructor. 


Table  19 

Student  Ratings  of  the  IC 
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Students  also  rated  the  overall  performance  of  the  IC  (Table  19).  Results  indicate  a  very 
positive  assessment  of  the  IC’s  role  in  the  classroom;  ratings  ranged  from  a  high  of  89%  ranking 
the  IC  as  Very  Good  or  Excellent  (95B10  and  TQL)  to  a  low  of  83%  (76Y10).  These  means 
reflect  a  very  positive  assessment  of  the  ICs.  Narrative  data  support  the  hypothesis  that  the  high 
ratings  of  the  IC  resulted  from  the  proximity  of  that  individual  to  the  students  in  the  classroom. 

Student  Ratings  of  the  Military  Site  Coordinator  (MSC) 

Students  were  asked  to  rate  how  helpful  the  MSC  was  in  helping  them  understand  the 
content  of  the  course  (Table  20).  There  were  no  data  reported  by  the  HazWaste  and  TQL 
courses  on  this  item  because  only  the  MOS  courses  had  a  MSC  in  the  classroom.  Though  all 
the  ratings  were  high,  100%  of  the  students  in  95B10  rated  the  MSC  as  Helpful  or  Very  Helpful 
in  aiding  them  to  understand  the  content  of  the  course.  Actually,  this  is  not  too  surprising.  In 
the  95B10  course,  the  MSC  conducted  the  off-line  physical  activities,  and  provided  a  frequent 
and  necessary  link  between  the  students  and  the  course  objectives. 

One  final  measure  is  the  students’  ratings  of  the  overall  effectiveness  of  the  MSC.  Again, 
ratings  for  the  MSC  in  all  courses  were  high.  At  the  least  85%  of  the  respondents  rated  the  MSC 
as  Very  Good  or  Excellent. 


Table  20 

Student  Ratings  of  the  MSC 
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In  summary,  students  were  generally  positive  about  the  instructional  team  and  found  the 
ICs  particularly  helpful.  In  the  narrative  data,  students  raised  questions  about  the  content 
expertise  of  the  civilians  but  they  were  otherwise  favorable  about  the  instructional  team’s 
effectiveness. 
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Conclusions 


In  examining  the  effectiveness  of  the  VTT  courses,  the  results  of  both  the  quantitative  and 
self-report  data  indicate  that  the  VTT  instruction  was  successful  in  helping  students  master  the 
learning  objectives.  The  results  indicate  that  significant  improvements  were  made  in  scores  from 
the  pretest  to  the  posttest  for  four  of  the  five  courses.  Students  in  the  fifth  course,  TQL,  were 
not  administered  a  pretest,  so  no  comparison  data  is  available.  In  addition,  the  average  number 
of  students  who  passed  the  PTs  on  the  first  attempt  was  90.03%  for  71L10,  97.99%  for  76Y10, 
and  94.5%  for  95B 10. 

The  background  variables  that  predicted  objective  performance  and  subjective  assessments 
of  achievement  (e.g.,  relevance  of  the  current  PMOS,  computer  skills)  were  those  that  would  be 
expected  to  predict  performance  for  any  type  of  course  delivery  medium.  Similarly,  the 
background  variables  did  not  tend  to  predict  ratings  of  course  materials  and  processes,  nor  overall 
course  ratings.  The  few  significant  correlations  that  were  obtained  could  likely  be  observed  in 
traditional  military  courses  as  well.  Thus,  these  results  suggest  that  there  is  no  need  to  structure 
VTT  courses  differently  for  different  student  populations.  Relatedly,  there  appears  to  be  no  need 
to  select  a  specified  population  of  students  for  VTT  courses  versus  traditionally  taught  courses. 

A  comparison  of  the  instructional  methods  used  in  the  various  courses  was  attempted. 
However,  the  instructional  methods  could  not  be  accurately  compared  because  there  were  so 
many  other  variations  in  the  courses  (e.g.,  total  number  of  hours,  FTXs,  etc.).  Since  all  students 
in  the  VTT  courses  met  criterion,  it  can  be  assumed  that  the  instructional  methods  used  in  the 
VTT  courses  were  effective. 

Both  students  and  instructional  personnel  rated  the  learning  methods,  interactivity,  and 
other  course  characteristics  developed  specifically  for  the  VTT  courses  to  be  generally  effective. 
Of  these  ratings,  the  amount  and  types  of  interactivity  were  the  most  highly  rated  (24  of  29 
ratings  were  at  4.0  or  above).  Time  to  cover  the  course  topics  was  consistently  rated  lowest. 
This  finding  is  consistent  with  the  distance  education  literature  that  states  that  VTT  courses  take 
longer  to  present  than  traditional  courses.  Students  rated  the  overall  quality  of  the  five  courses 
as  3.19  (95B10),  3.55  (76Y10),  3.59  (71L10),  4.04  (HazWaste)  and  4.10  (TQL),  indicating  that 
students  thought  the  courses  ranged  from  slightly  better  than  good  to  slightly  better  than  very 
good. 

The  data  gathered  from  a  variety  of  observations  indicated  students  had  a  high  level  of 
interest  in  the  VTT  system  of  instruction  and  a  very  favorable  attitude  toward  taking  future 
military  classes  via  VTT  at  a  community  college  site.  Therefore,  this  method  of  instruction 
appears  to  be  an  effective  option  when  considering  the  delivery  of  training  to  remote  sites. 

Fifteen  different  evaluation  items  rated  the  effectiveness  of  the  instructional  team  from 
the  students’  perspectives.  Most  (67%)  of  the  measures  of  the  instructional  team  focused  on  the 
civilian  VTT  instructor,  with  the  balance  of  the  questions  addressing  the  effectiveness  of  the  IC, 
MSC,  and  the  MIA.  ICs,  MSCs,  and  MIAs  were  evaluated  on  their  interactions  with  students 
in  the  classroom  and  their  overall  performance.  Seventy  percent  of  these  were  above  4.0, 
reflecting  the  respondents’  positive  reactions  to  members  of  the  team.  Of  the  remaining  ratings, 
28%  were  between  3.0  and  4.0,  and  only  2%  were  below  3.0. 
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In  summary,  looking  at  demographic  data,  it  appears  that  VTT  is  an  acceptable  approach 
for  the  general  military  population,  because  this  data  predicted  success  in  the  VTT  courses  as 
similar  data  would  have  predicted  success  in  traditional  courses.  Thus,  standard  military 
requirements  and  course  prerequisites  are  adequate  for  selecting  students  for  VTT  courses.  Also, 
100%  of  the  students  in  the  MOS  courses  mastered  the  learning  objectives  specified  in  the 
courses. 

Issues 

When  designing  VTT  courses,  appropriate  VTT  instructional  strategies  and  methods, 
including  multiple  chances  for  interaction,  must  be  designed  into  the  instruction.  In  addition, 
attention  must  be  focused  on  course  design  and  development  to  ensure  that  each  course  takes 
advantage  of  the  VTT  environment,  and  that  the  components  of  the  course  are  consistent.  Thus, 
one  of  the  key  tenets  of  effective  VTT  is  that  the  instruction  must  be  more  carefully  planned  than 
traditional  instruction. 

This  planning  should  also  take  into  consideration  the  fact  that  it  takes  longer  to  teach 
using  a  teletraining  system  than  it  does  in  a  standard  classroom.  This  means  that  platform 
courses  converted  to  VTT  will  require  either  a  longer  training  period  to  cover  the  same  topics 
or  that  some  topics  or  activities  must  be  shortened  or  deleted  from  the  instruction. 

The  proponent  schools  for  71L10,  76Y10,  and  9SB10  may  be  able  to  use  the  data  on  the 
numbers  of  students  who  passed  the  PTs  on  the  first  attempt  as  a  way  to  determine  whether 
certain  tasks  in  their  curricula  might  need  to  be  redesigned  or  revised.  The  proponent  schools 
might  use  these  data  to  analyze  both  the  instruction  and  the  PTs  where  a  large  percentage  of 
students  had  to  retake  the  PT  to  reach  mastery. 

Word  pictures  and  graphics  may  lose  their  effectiveness  in  longer  courses,  perhaps 
especially  when  they  are  used  extensively,  thus  reducing  their  novelty.  Comments  from  a  few 
of  the  95B10  and  TQL  students  were  that  "word  pictures  can  seem  too  low-level  and  simplistic 
if  care  is  not  taken  to  make  them  meaningful."  Use  of  word  pictures  may  vary  according  to  the 
educational  level  of  the  students.  However,  the  fact  that  the  word  pictures  and  graphics  were 
rated  as  helpful  (3.0)  may  mean  that  students  do  need  the  focus  that  word  pictures  provide. 

While  opportunities  to  interact  with  students  at  other  sites  were  built  into  the  instruction, 
there  were  fewer  opportunities  for  this  interaction  because  such  interaction  was  time  consuming 
and  the  audio  protocols  for  talking  to  each  other  across  sites  were  more  complicated  than  simply 
talking  to  the  instructor.  Due  to  technical  characteristics  of  the  TNET  equipment  it  is  doubtful 
that  the  audio  protocols  could  be  modified  very  much.  However,  when  using  TNET  in  the 
future,  close  attention  needs  to  be  paid  to  how  to  maximize  interaction  among  students  over  the 
network. 

In  order  to  assess  the  effectiveness  of  VTT  courses  with  corresponding  traditional  courses, 
student  performance  data  must  be  available  regarding  both  methods  of  instruction  and 
performance  results.  However,  the  FTP  did  not  have  access  to  this  data  and  thus  an  empirical 
investigation  comparing  subjective  ratings  and  actual  performance  in  the  VTT  courses  with  their 
traditional  counterparts  could  not  be  conducted.  For  future  studies,  this  data  must  be  available 


for  any  comparisons  to  be  made.  In  general,  however,  past  research  has  shown  that  VTT 
students  master  the  learning  objectives  as  do  students  in  traditional  courses. 

Another  issue  is  that  an  assessment  of  the  instructional  team’s  effectiveness  could  be 
broadened  by  administering  more  objective  evaluation  items  to  instructional  team  members. 
These  items  could  be  framed  much  like  the  evaluations  given  to  the  students.  Though  the  sample 
size  would  be  small  (limited  to  one  instructional  role  (IC,  MIA,  MSC)  for  each  course  at  a 
remote  site),  additional  quantitative  data  on  the  instructor  and  other  team  members  may  provide 
useful  information.  In  addition,  the  Instructional  Manager,  Instructional  Designers,  Instructional 
Production  Managers  and  Instructional  Technology  Specialists  would  probably  have  valuable 
information  about  the  effectiveness  of  the  instructional  team. 

Success  of  the  FTP  was  due  to  careful  planning  of  the  project.  For  example,  contingency 
plans  were  developed  so  that  no  instruction  would  be  missed  due  to  equipment  failure, 
instructional  personnel  were  well  trained  for  their  respective  roles,  there  was  excellent  support 
from  military  organizations  and  personnel,  and  there  were  coordinated  efforts  of  community 
college,  military  instructors,  and  support  personnel.  It  is  expected  that  any  VTT  course  would 
require  a  similar  effort. 
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OBJECTIVES  RELATED  TO  THE  COMMUNITY  COLLEGES 


Community  Colleges  as  Providers  of  Military  Instruction 

An  increasing  need  for  educational  services  for  the  reserve  forces  has  prompted  local 
colleges  to  become  providers  of  military  training.  This  need  is  due  to:  (a)  the  fact  that  reserve 
units  often  need  additional  or  new  training  because  they  have  received  new  equipment,  been 
reorganized,  or  have  changed  missions  (Watt,  1988),  and  (b)  the  fact  that  the  educational 
standards  for  service  members  have  been  raised.  The  Army  Regulation  (AR)  621-5  sets  the 
minimum  goals  for  members  of  the  National  Guard  and  the  U.S.  Army  Reserve.  By  1  October 
1989  all  enlisted  persons  in  the  Army  RC  were  required  to  have  a  high  school  diploma  or  a 
GED.  Enlisted  personnel  and  Warrant  Officers  are  required  to  have  an  associate  degree  by  their 
fifteenth  year  of  reserve  service;  commissioned  officers  must  hold  a  baccalaureate  degree  in  order 
to  be  promoted  to  grade  04  by  their  eighth  year  of  service. 

These  educational  needs  require  new  solutions  to  military  training  options.  Watt  (1988) 
reported  that  the  National  Guard  Bureau  recently  called  a  roundtable  meeting  to  promote  more 
and  stronger  links  between  local  colleges  and  National  Guard  units.  This  upper  level  military 
support  accompanied  by  programs  that  help  defray  the  costs  of  tuition,  e.g.,  the  "Montgomery" 
GI  Bill  and  the  Army  Continuing  Education  Support  (ACES)  program,  along  with  other  more 
traditional  financial  support  (e.g.,  student  loans),  is  making  the  local  colleges  a  very  attractive 
place  for  soldiers  to  attain  college  credit  and  degrees. 

The  educational  needs  of  the  RC  typically  fall  into  one  of  three  major  areas: 
developmental  studies,  military  skill  training  at  the  unit  level,  and  individual  and  continuing 
higher  education  (Watt,  1988).  Watt  reported  that  there  are  several  educational  programs  that 
link  the  military  and  local  colleges  to  provide  training  in  these  three  areas.  While  these  college 
and  military  educational  partnerships  were  varied  in  terms  of  the  extensiveness  of  the  programs 
offered,  the  number  of  credits  awarded,  and  how  tuition,  fees  and  expenses  were  paid.  Watt 
(1988)  reported  that  "even  with  the  cost  of  paying  for  full  time  duty,  the  expense  for  the  Guard 
is  less  than  to  send  a  person  to  an  active  duty  Army  service  school.  Moveover,  the  Guard  is 
finding  it  easier  to  get  companies  to  release  their  employees  for  a  year  to  go  to  a  local  college 
than  to  send  them  ’off  somewhere  to  an  Army  post  for  the  same  period  of  time"  (p.  8).  Some 
examples  of  these  programs  are: 

•  The  U.S.  Navy  Reserve  recruits  a  qualified  person  for  some  medical  units  and 
sends  the  reservists  to  a  local  college  that  has  an  approved  program  in  the  needed 
field. 

•  A  local  college  in  a  midwestem  state  taught  a  192-hour  course  at  the  unit’s  drill 
site  in  each  of  three  job  skill  areas  (carpenters,  plumbers,  and  electricians)  because 
the  reserve  unit  construction  team  needed  the  training  and  could  not  provide  it 
themselves. 

•  A  Reserve  Forces  School  and  a  two-year  college  in  the  northwestern  part  of  the 
U.S.  started  a  partnership  to  teach  classes  in  utilities  and  power  generation  repair. 
The  students  received  military  certification  and  college  credits. 
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•  A  technical  college  in  South  Carolina  designed  and  developed  a  one-year 
certification  program  for  National  Guard  personnel  in  precision  measuring 
equipment.  Students  earned  73  quarter  hours  of  credit  and  were  awarded  the 
appropriate  military  occupational  specialty. 

•  A  college  in  Tennessee  conducted  computer  orientation  classes  for  the  state’s 
National  Guard  using  their  equipment  and  software. 

Watt  suggested  that  local  colleges  with  the  greatest  number  of  innovations  will  be  the 
most  successful  providers  of  military  training.  Some  of  the  most  important  considerations  are: 

•  "using  the  armory  or  drill  hall  as  the  location  for  class  delivery; 

•  scheduling  classes  at  times  more  convenient  for  the  reservists,  e.g.,  the  evening 
prior  to  a  weekend  drill  or  the  evening  between  the  two  days  of  a  weekend  drill; 

•  offering  courses  in  formats  other  than  the  usual  classroom  mode,  e.g.,  cable 
television; 

•  accepting  comparable  course  completion  through  use  of  CLEP  or  locally  devised 
tests"  (p.  12-13). 

In  summary,  the  need  for  military  units  to  retrain,  for  military  personnel  to  earn  associate 
and  bachelor  degrees,  and  the  reservists’  limited  training  availability  (a  reservist  is  only  available 
for  the  equivalent  of  thirty-eight  calendar  days  a  year  (Watt,  1988))  makes  local  colleges  and 
military  units  a  natural  partnership.  Watt  stated  that  the  reserve  forces  are  an  underserved 
educational  population  and  that  local  educational  institutions  can  meet  many  of  the  reservists’ 
needs.  Evidence  exists  that  such  partnerships  can  work  and  that  they  provide  many  benefits  to 
both  the  college  and  the  reserve  unit.  The  FTP  was  designed  in  part  to  test  the  capability  of  the 
community  colleges  to  provide  VTT  instruction  to  reservists. 


Objective  Cl 

Assess  the  Existing  Technical  Capability  of  the  Community  Colleges 
to  Provide  VTT  Instruction  (Facilities,  Equipment,  Personnel) 


A  review  of  the  facilities,  equipment,  and  personnel  at  the  origination  site  (FCCJ)  and  the 
remote  sites  (VCC,  SPJC,  and  FCCJ2)  was  necessary  to  determine  their  technical  capabilities  in 
delivering  the  VTT  instruction.  Analysis  of  these  capabilities  is  critical  in  calculating  two 
important  variables:  (a)  the  number  of  students  that  can  be  accommodated  through  the  program, 
and  (b)  the  cost  of  providing  all  community  college  sites  with  the  equipment  and  technicians  to 
successfully  deliver  the  instruction. 

This  objective  regarding  the  technical  capability  and  the  next  one  (C2)  that  assesses  the 
instructional  capability  of  community  colleges  are  two  sides  of  the  same  coin.  One  could 
effectively  argue  that  technical  capability  is  included  in  the  description  of  instructional  capability. 
Electronic  media  and  other  information  delivery  systems  are  essential  elements  of  instructional 
packages.  However,  technical  capability  is  required  to  design  and  deliver  courseware,  e.g.,  to 
produce  graphics  and  videotapes,  and  can  therefore  be  addressed  independently  of  instructional 
capabilities.  The  scope  of  the  FTP  required  the  evaluators  to  analyze  segments  of  the  community 
colleges’  capabilities.  Additionally,  both  the  technical  and  instructional  capabilities  can  be 
evaluated,  partly  at  least,  by  the  types  of  staff  training  that  were  required  for  conducting  the 
project.  Staff  training  is  addressed  in  Objective  C3. 

In  discussing  this  objective  it  is  important  to  understand  FCCJ’s  position  in  the  state 
regarding  instructional  TV  resources.  FCCJ  is  one  of  five  (of  28)  community  colleges  in  Florida 
to  have  extensive  instructional  video  production  facilities  and  equipment.  SPJC  and  VCC  were 
not  included  in  this  group  of  five,  and  their  transmit  and  receive  capacities  were  provided  only 
by  the  TNET  equipment.  It  was  therefore  neither  expensive  nor  difficult  to  upgrade  the  video 
capabilities  at  FCCJ  to  serve  as  the  origination  and  production  site  to  conform  with  the  course 
design  and  TNET  requirements.  All  three  sites  were  staffed  with  experienced  personnel  to 
support  the  technical  needs  of  the  project.  In  most  cases,  there  was  no  need  to  hire  new  people 
to  fulfill  that  role;  existing  personnel  were  reassigned  from  their  regular  duties  to  cover  the  FTP 
needs. 

FCCJ  is  typical  of  community  colleges  involved  in  instructional  video  production  in  that 
it  has  a  full  production  and  post-production  facility.  The  production  equipment  included  a 
broadcast-quality  studio  and  portable  cameras,  sound  equipment,  and  a  special  effects  generator 
to  supplement  the  editing  video  recorders.  Personnel  included  the  production  manager,  chief 
engineer,  and  various  production  support  personnel.  The  Instructional  Television  Manager  at 
FCCJ  provided  a  list  of  equipment,  facilities  and  personnel  at  the  origination  and  remote  sites, 
as  well  as  a  list  of  what  was  added  to  meet  VTT  requirements.  Table  21  lists  the  facilities, 
equipment,  and  personnel  that  existed  at  the  origination  site  when  the  project  started  and  what 
was  added  or  modified  to  FCCJ  in  order  to  produce  and  present  TNET  instruction.  Neither  SPJC 
nor  VCC  had  the  technical  capabilities  to  produce  the  instruction. 
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Table  21 

Origination  Site:  FCCJ 


A.  What  existed  in  the  TV  production  facility  prior  to  FTP: 

Facility:  TV  studio  with  full  production  and  post-production  capabilities 
Production  Equipment: 

•  2  Sony  M-7  chip  cameras 

•  Sony  SP  portable  record  decks 

•  Lights,  wireless  microphones,  and  other  standard  ancillary  field  equipment,  such  as 
monitors 

•  Consumables,  such  as  tape  stock  and  batteries 

•  Teleprompter  (in  studio  productions) 

Post-production  Equipment: 

•  Sony  SP  editor  and  decks 

•  CEL  for  DVE  (Digital  Video  Effects) 

•  Ross  Switcher 

•  Amiga  2S00  (Toaster)  for  character  generation  and  some  animation 

•  Compact  Disc  (CD)  player  and  cleared  music  library  were  used  for  music  source 
Personnel: 

•  TV  Production  Manager 

•  Chief  Engineer 

•  A-V  Technicians 

•  Administrative  Support  Personnel 

•  Production  Staff: 

•  Cameraperson/Editor 

•  Computer  Graphics  Specialist 

B.  Technical  Requirements  Added  for  VTT  Instruction: 

Facility: 

•  Video  segments  and  computer  graphics  were  designed  into  the  instruction,  but  a 
TNET  course  could  be  presented  without  them.  For  the  FTP,  the  following  was 
needed: 

•  Production  and  post-production  capabilities  for  off-site  shooting 

•  Computer  graphics  generation  for  editing 

•  FCCJ  facility  was  modified  in  the  following  ways: 

•  Added  back-up  generator  for  electrical  power 

•  Two  analog  telephone  lines  had  to  be  added 

•  Cellular  phone  in  case  of  power  outage 

Equipment:  With  the  exception  of  the  TNET  gallery,  all  the  following  equipment  existed 
at  FCCJ,  but  was  needed  and  was  supplemental  to  TNET: 

•  A  VHS  tape  player  and  audio  cable  connected  to  TNET 

•  Additional  lights 

•  A  lavaliere  microphone 

•  A  5"  monitor  connected  to  the  ELMO  allowing  the  technical  assistant  to  focus  and 

center  visuals  before  displaying  them 

•  An  audio  cassette  deck  to  transmit  pre-recorded  music  over  the  network 

•  A  set  used  as  a  backdrop  to  the  VTT  instructors  with  appropriate  course-related 
logo  mounted  and  hung  on  the  set  framework  behind  the  instructor 

Personnel: 

•  Network  Production  Assistant 

•  Existing  personnel  were  reassigned  from  their  regular  duties 
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The  standard  TNET  classroom  equipment  package  from  AETD  was  added  at  all  sites 
(Appendix  H),  ensuring  that  any  of  the  sites  could  function  as  an  origination  site.  The  number 
of  students  at  each  site  was  limited  by  factors  related  to  the  technology.  No  more  than  15 
students  were  assigned  to  a  classroom  in  order  to:  (a)  give  everyone  a  clear  view  of  the  TV 
monitor,  (b)  for  everyone  to  be  seen  well  by  the  classroom  camera,  and  (c)  to  control  the  amount 
of  time-consuming  interaction  with  the  instructor.  While  a  maximum  of  15  students  is  not  a 
TNET  directive,  FTP  program  designers  thought  more  than  fifteen  students  in  the  class  would 
inhibit  opportunities  for  questions  or  responses  from  students,  and  cause  the  course  to  run  longer 
than  the  prescribed  period  of  time.  Although  seven  receive  sites  was  considered  the  maximum 
advisable  with  the  TNET  system,  only  five  receive  sites  were  in  operation  at  any  one  time. 
Table  22  lists  the  facilities,  equipment,  and  personnel  at  the  remote  sites.  Table  23  lists  the  roles 
and  responsibilities  of  the  technical  personnel  at  the  origination  and  remote  sites. 

The  remote  site  at  FCCJ  (FCCJ2)  was  a  special  case  because  this  site  was  used  as  an 
official  visitation  center  during  the  project  and  as  field  test  or  intensive  site  during  the  MOS 
courses.  The  classroom  was  modified  for  this  purpose,  e.g.,  installing  a  one-way  mirror  for 
visitors.  These  modifications  are  included  in  Table  24. 

At  the  end  of  the  instruction  all  of  the  TNET  classroom  equipment  was  removed  from  the 
sites  and  returned  to  the  government. 
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Table  22 

Remote  Sites:  SPJC  and  VCC 


A.  What  existed  prior  to  FTP: 

Facility: 

•  Basic  classroom  designated  for  TNET  use  (SPJC  converted  space  for  classroom  use  to 
meet  the  requirements) 

•  Size  and  temperature  requirements  for  TNET  were  met 

•  Tables  and  chairs  for  regular  classroom  seating 

Equipment: 

•  VCR  for  remedial  instruction  and  contingencies  available  for  use  as  needed 
Personnel: 

•  A-V/Television  coordinator  at  each  site 

B.  What  was  Added/Needed  for  VTT  Instruction: 

Facility: 

•  Special  wiring  for  TNET 

•  Close  to  smoking/snack/resiroom  facilities 

•  95B10:  space  for  hands-on  practical  exercises 

Equipment: 

•  Standard  TNET  Classroom  equipment 

•  Telephone 

•  Access  to  Fax 

•  Filing  and  storage  areas/cabinets 

•  Scale  for  weighing  students  in  at  the  beginning  of  each  MOS  course 

•  71L10:  Typewriters 

•  76Y10:  Microfiche  readers 

•  95B10:  MP  course  related  equipment 

Personnel: 

•  Hired  Site  Technicians  (Backup) 

•  Facilities  Coordinator  -  reassigned  from  regular  duties 

•  Administrative  POC  --  reassigned  from  regular  duties 
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Technical  assistance  required  for  course  development 


Table  24 

Remote  Site:  FCCJ2  --  Intensive  Site 


A.  What  existed  prior  to  FTP: 

Facility : 

•  Same  as  other  remote  sites 
Equipment: 

•  Same  as  other  remote  sites  except  that  a  small  room  or  office  adjacent  to 
the  classroom  was  needed  as  a  viewing/visitor  area 

Personnel: 

•  Same  as  other  remote  sites 

B.  What  was  Added/Needed  for  VTT  Instruction: 

Facility: 

•  Added: 

•  one-way  mirror/glass  for  viewing  by  visitors 

•  viewing  area  was  wired  for  sound 
Equipment: 

•  Added: 

•  TV  camera  and  tape  deck  to  record  instruction  for  remedial  and 
contingency  purposes 

•  Telephone  in  observation  room 
Personnel: 

•  Same  as  other  remote  sites 


Results  of  the  Data 

There  are  only  a  few  evaluation  measures  of  the  community  colleges’  technical 
capabilities.  In  one  example,  students  in  all  courses  rated  the  performance  of  the  technician 
working  the  equipment  on  a  5-point  scale,  with  Excellent  being  the  highest  rating  (5)  and  Poor 
being  the  lowest  (1).  Table  25  presents  these  results.  As  can  be  seen  from  the  data,  the 
technicians  for  all  five  courses  were  consistently  rated  as  better  than  very  good. 


Table  25 

Student  Ratings:  Performance  of  Technician  Working  Equipment 
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Administrative  Point-of-Contact  (POC) 


Another  source  of  information  regarding  community  college  technical  support  was  the 
end-of-course  review  provided  by  the  PCX!.  The  POC  was  responsible  for  overseeing  and 
coordinating  the  project  activities  at  each  community  college  remote  site.  Each  POC  was  asked 
to  rate  how  difficult  it  was  to  perform  various  project  tasks:  to  find  a  suitable  classroom, 
equipment,  and  to  find  staff  to  administer  the  courses.  Very  Easy  tasks  were  rated  5;  very 
difficult  tasks  were  rated  1.  These  results  are  presented  in  Table  26.  The  data  indicate  that  POCs 
found  little  difficulty  in  securing  site  commitments  on  various  technical  variables.  Note, 
however,  that  a  small  sample  size  on  these  items  makes  it  inappropriate  to  make  broad 
generalizations  from  the  data. 


Table  26 

POC  Ratings:  Difficulty  in  Securing  Site  Commitments 
(MOS  Courses  Only) 
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Project  Evaluators 

One  other  measure  of  community  college  technical  capability  was  a  checklist  of  effective 
course  delivery  conditions  completed  by  the  project  evaluator  at  each  site  on  the  first  day  of 
instruction  for  each  of  the  MOS  courses.  This  checklist  included  a  number  of  technical  and 
personnel  items,  such  as  facilities,  equipment,  and  community  college  support.  No  statistical  data 
were  generated  from  this  evaluation.  If  a  problem  occurred  a  corresponding  category  was 
checked.  However,  narrative  data  related  to  this  checklist  indicated  that  all  the  technical 
requirements  for  VTT  at  each  community  college,  and  all  back-up  procedures,  were  available  and 
ready  for  use  on  the  first  day  of  instruction  for  all  the  courses  for  all  the  sites. 

In  summary,  this  objective  addressed  the  technical  capability  of  the  community  colleges 
to  provide  VTT  instruction,  theorizing  that  this  technical  capacity  would  affect  the  number  of 
students  that  could  be  processed  through  the  courses.  While  technical  constraints  did  limit  the 
number  of  students,  this  restriction  was  more  a  function  of  course  design  issues  and  TNET 
requirements  than  the  community  colleges’  equipment,  facilities  and  personnel.  A  review  of  the 
equipment  and  personnel  required  for  TNET  indicated  that  each  community  college  needed  to 
provide  technical  resources  to  support  VTT  instruction,  and  that  they  were  able  to  do  so. 
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Objective  C2 

Assess  the  Existing  Instructional  Capability  of  the  Community  Colleges 
Related  to  Military  Course  Development  and  Delivery 
(Expertise,  Personnel,  Instructional  Resources) 

The  purpose  of  this  objective  was  to  describe  the  instructional,  rather  than  the  technical 
resources  of  community  colleges,  focusing  on  their  strengths  and  weaknesses  relative  to 
designing,  developing,  and  delivering  the  instruction.  For  example,  to  what  extent  do  the  faculty 
have  the  knowledge  bases  to  design  and  deliver  military  instruction?  Community  colleges 
traditionally  use  many  adjunct  faculty  members  trained  to  teach  a  variety  of  specific  topics 
covering  a  wide  range  of  interests.  These  instructors,  plus  the  regular  staff,  may  comprise  a  pool 
of  talent  from  which  individuals  can  be  drawn  to  staff  innovative  courses. 

According  to  the  Project  Director  at  FCCJ,  the  mission  of  the  community  college  is  to 
offer  viable  educational  opportunities  to  all  members  of  the  surrounding  community;  these 
opportunities  are  not  available  from  traditional  four-year  institutions  of  higher  learning. 
Community  colleges  have  forged  bonds  with  private  industry  and  public  agencies  designing 
programs  and  curricula  in  response  to  specific  needs.  VCC  in  Orlando  describes  itself  in  its 
catalogue  as  an  imaginative  institution  of  higher  education;  it  offers  courses,  seminars  and 
workshops  at  multiple  campuses  and  centers  utilizing  traditional  class  curricula,  as  well  as 
emerging  instructional  technologies.  SPJC  contends  in  the  foreword  to  its  catalogue  that  it  seeks 
to  respond  to  changing  educational  needs  with  flexibility  and  innovation  in  the  way  services  are 
provided,  providing  technical  assistance,  customized  training,  and  other  professional  services  for 
business  and  industry. 

Additionally,  the  growth  of  community  colleges  has  given  most  major  communities  a 
chance  to  house  a  college  or  a  network  of  urban  and  suburban  campuses.  Community  colleges 
are  now  within  a  reasonable  distance  for  most  people,  providing  them  with  ready  access  to 
college  programs  and  resources.  Community  colleges  have  also  been  innovative  in  selecting 
unorthodox  sites  for  outreach  campuses.  For  example,  FCCJ  uses  a  large  section  of  a  suburban 
mall  as  classroom  space  for  its  students.  The  combination  of  a  skilled  and  diverse  instructional 
staff,  current  industrial  technology,  and  a  broad  geographical  dispersion  make  the  community 
college  a  natural  location  for  projects  with  uncommon  requirements.  The  campuses  appear  to 
be  uniquely  qualified  to  provide  instructional  and  logistical  support  for  a  program  like  the  FTP. 

Results 

Information  assessing  the  instructional  capabilities  of  community  colleges  was  not 
available  in  objective  measures  but  was  found  in  the  narrative  data  gathered  from  administrators, 
instructors,  and  evaluators.  Responses  were  broadly  categorized  into  three  areas;  types  of 
instructional  personnel  needed  for  a  project  such  as  the  FTP,  expertise  of  personnel,  and 
instructional  resources. 

Instructional  Personnel.  There  were  two  primary  aspects  of  the  instructional  component 
of  the  FTP:  (a)  design  and  development  of  the  instruction  and  (b)  delivery  or  implementation 
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of  the  instruction.  The  community  college  instructional  personnel  required  for  the  FTP  were: 
VTT  instructors/course  developers,  ICs  at  the  remote  sites,  and  an  instructional  manager.  The 
VTT  instructors/course  developers  and  the  instructional  manager  were  community  college  faculty 
from  FCCJ;  the  ICs  were  community  college  faculty  at  the  remote  sites.  The  primary  selection 
criteria  for  the  VTT  instructors/course  developers  and  ICs  was  content  expertise;  the  primary 
selection  criteria  for  the  instructional  manager  was  broad  instructional  and  technical  expertise. 

An  administrative  decision  was  made  that  the  course  designers  would  also  serve  as  the 
VTT  instructors  and  present  the  courses  to  the  students.  This  was  not  an  arbitrary  decision 
because  it  was  assumed  that  the  individual  who  designed  the  instruction  would  be  better  prepared 
to  deliver  it.  Since  the  timeframe  for  design  and  development  was  limited,  the  course  developers 
served  as  the  on-camera  VTT  instructors.  However,  another  option  would  have  been  to  select 
one  group  of  faculty  to  develop  the  courses,  and  a  different  group  to  deliver  the  instruction. 

In  addition  to  the  VTT  instructors/course  developer,  another  person  needed  for  the 
instructional  team  was  an  instructional  manager.  This  person  was  responsible  for  coordinating 
the  instructional  and  technical  components  of  the  courses.  He  was  also  the  liaison  between  the 
Instructional  Television  Manager  at  FCCJ  and  the  ICs. 

The  ICs  were  selected  to  be  involved  in  the  project  by  the  community  college  where  they 
were  a  faculty  member.  They  were  selected  based  on  their  subject  matter  or  content  expertise 
for  a  particular  course.  The  roles  and  responsibilities  of  the  VTT  instructors  and  the  ICs  are 
listed  in  Table  27. 

Expertise.  All  instructional  personnel  for  the  FTP  were  professors,  either  full  time  or 
adjunct,  at  the  community  colleges.  At  the  origination  site,  four  of  the  five  VTT  instructors  were 
faculty  members  selected  from  departments  closely  associated  with  the  course  they  would  be 
teaching.  For  example,  the  71L10  instructor  was  a  professor  of  Office  Systems  Technology.  The 
exception  was  the  76Y10  instructor  who  was  a  professor  of  English.  Since  there  were  no 
departments  or  programs  that  approximated  the  content  of  the  76Y10  course.  Unit  Supply 
Specialist,  this  professor  was  selected  because  of  his  previous  course  development  experience  and 
teaching  ability. 

Community  college  faculty  are  professional  educators.  As  such  they  are  typically  good 
teachers.  However,  part  of  their  role  for  the  FTP  was  to  develop  as  well  as  present  the  courses. 
While  all  of  them  had  designed  numerous  courses  in  their  own  fields,  none  of  them  were 
professional  instructional  designers.  Therefore,  they  did  not  know  the  SAT  model  in  its  entirety. 
They  were,  however,  knowledgeable  about  components  of  the  SAT,  e.g.,  behavioral  objectives. 


96 


Table  27 

Roles  and  Responsibilities  of  Instructional  P 
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It  is  not  unusual  for  faculty  members  at  community  colleges  or  at  four-year  institutions 
to  be  excellent  teachers,  but  not  to  be  course  developers  in  the  structured  sense  of  the  SAT.  The 
VTT  course  development  process  required  a  precision  that  is  not  typically  used  when  conducting 
platform  instruction.  To  eliminate  this  deficiency,  staff  training  about  the  SAT  course 
development  was  presented  to  the  course  developers. 

The  Instructional  Manager  at  the  origination  site  was  a  professor  of  Computer/Information 
Systems.  His  expertise  in  computers  was  invaluable  in  using  the  computer  software  needed  to 
develop  the  graphics,  and  in  interfacing  the  computer  and  VTT  equipment.  While  this  professor 
did  not  do  any  of  the  VTT  teaching,  a  component  of  his  role  was  to  oversee  the  design  and 
development  of  the  courses.  Consistent  with  the  skills  of  the  other  professors,  he  was  not  a 
professional  course  developer  proficient  in  the  use  of  the  SAT  model.  As  a  consequence,  FCCJ 
hired  a  consultant  who  was  a  professional  instructional  designer  to  assume  the  role  to  work  with 
the  faculty  as  they  developed  the  VTT  courses. 

The  ICs  at  the  remote  sites  were  also  either  full-time  or  adjunct  faculty  members  at  their 
respective  institutions.  With  the  exception  of  the  76Y10  course,  all  ICs  were  selected  from 
departments  or  programs  closely  allied  with  the  course  being  delivered.  This  was  especially 
necessary  for  the  71L10  course  because  the  IC  taught  the  typing  component  of  the  course.  For 
the  95B10,  Basic  Military  Police,  the  ICs  were  proficient  in  civilian  law  enforcement.  Due  to 
a  military  student  certification  requirement,  the  MSCs  for  95B10  were  certified  MP  instructors. 
However,  because  the  students  also  received  college  credit,  the  95B10  ICs  were  professors  from 
related  disciplines. 

Only  one  of  the  VTT  instructors  and  only  a  few  of  the  ICs  had  military  expertise  or 
experience.  The  95B10  instructor  had  at  one  time  held  the  MOS  in  Basic  Military  Police  and 
one  other  VTT  instructor  had  taught  a  course  for  the  military.  Several  ICs  had  military 
experience,  but  none  of  them  held  the  MOS  or  certification  in  the  courses  for  which  they  were 
assigned.  This  lack  of  military  experience  prompted  the  FTP  to:  (a)  hire  a  military  training 
specialist  and  (b)  obtain  military  SMEs  in  each  of  the  five  courses  that  were  being  delivered. 

In  summary,  while  the  four  of  the  five  faculty  at  FCCJ  were  experienced  teachers  in  the 
content  area  of  the  course  they  were  designing  and  delivering,  none  of  them  had  VTT  experience, 
they  had  very  limited  instructional  design  experience,  and  limited  military  experience  or 
background.  At  the  remote  sites,  with  the  exception  of  the  76Y10  faculty,  all  were  selected  from 
related  content  areas  for  the  course  that  they  were  participating  in  and  they  were  also  experienced 
community  college  teachers.  (While  they  were  not  community  college  faculty,  it  is  important 
to  note  that  the  support  personnel  at  the  Rhode  Island  and  Alabama  sites  did  not  have  content 
expertise  in  HazWaste  and  TQL). 

Instructional  Resources 

Some  of  the  instructional  resources  (e.g.,  typewriters,  microfiche,  videotape  recorders  etc.) 
were  previously  listed  in  the  tables  included  in  Objective  Cl.  These  technical  resources  were 


98 


used  for  instructional  purposes,  but  were  maintained  and  often  supplied  by  the  technical  staff. 
A  few  additional  comments  about  instructional  resources  are  inducted  here. 

Personnel  at  all  campuses  provided  the  space  as  specified  by  the  TNET  requirements  for 
the  program.  Inevitably,  some  campuses  had  better  facilities  than  others;  for  instance,  the  small 
FCCJ2  classroom  often  restricted  movement  for  some  in-class  activities  such  as  typing  and  the 
95B10  PEs.  In  the  71L10  course,  poor  typewriter  maintenance  created  problems  in  getting  all 
the  students  to  the  MOS  required  typing  speed.  The  9SB10  course  had  special  requirements  for 
its  hands-on  training,  and  some  sites  were  better  suited  than  others.  For  example,  students  at 
SPJC  were  at  a  modem  criminal  justice  training  facility,  and  they  praised  the  quality  of  the 
equipment  and  resources.  Students  at  FCCJ2  were  less  positive  in  their  assessment  of  the 
facility. 

One  final  facility  problem  revolved  around  the  all-week  instructional  schedule. 
Coordinators  found  that  many  college  support  services,  such  as  access  to  libraries  and  copy  and 
fax  machines,  were  unavailable  during  the  weekend  classes.  The  problems  were  solved  as  needed 
for  the  project,  but  in  the  future,  attention  to  the  instructional  support  requirements  for  the 
courses  should  be  carefully  considered. 

Despite  the  problems,  personnel  at  the  community  colleges  were  unanimous  in  supporting 
more  joint  efforts  between  the  military  training  establishment  and  community  colleges.  Each 
group  would  like  to  design  courses  to  fit  its  own  needs  more  closely,  for  example,  civilians 
would  like  college  credits  awarded  more  easily,  and  the  military  would  prefer  instructors  who 
are  MOS  qualified,  but  all  would  like  the  opportunity  to  work  with  future  programs.  As  the  POC 
at  VCC  said,  "It’s  a  win-win  situation  for  everybody." 

Student  Responses 

Students  were  asked  if  they  thought  there  were  advantages  to  having  the  course  offered 
at  a  community  college  instead  of  a  military  institution.  The  results  are  presented  in  Table  28. 
Ninety-seven  percent  (N=96)  of  the  students  responded  to  the  question.  Eighty-two  percent 
(N=79)  considered  the  community  college  atmosphere  preferable  to  that  of  the  military 
classroom,  while  18%  (N=17)  considered  the  college  classroom  no  different  or  not  as  good  as 
a  military  setting.  White  this  item  does  not  specifically  address  the  instructional  capabilities  of 
community  colleges,  it  does  approach  the  concept  indirectly.  The  advantages  of  the  community 
college  site  reflects  in  part  the  instructional  capabilities  of  the  institution. 
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Table  28 

Student  Ratings:  Advantages  of  Community  College  Sites 


SPJC 

87%  (26) 

13%  (4) 

vcc 

97%  (31) 

3%  (1) 

FCCJ2 

65%  (22) 

35%  (12) 

Total 

82%  (79) 

18%  (17) 

In  summary,  there  was  little  information  available  from  the  course  evaluations  that  directly 
addressed  this  objective.  A  conceptual  analysis  of  the  community  colleges’  faculties’  skills  and 
knowledge  bases  was  the  primary  input  for  this  objective.  With  their  comprehensive  missions 
and  variety  of  flexible  academic  programs,  community  colleges  appear  to  have  the  resources  to 
adjust  to  the  specific  instructional  needs  of  military  training.  The  faculty  are  experienced 
teachers  and  have  some  informal  course  development  skills.  They  are  typically  lacking  in 
military  expertise.  However,  given  sufficient  time  for  planning  and  staff  training,  the  community 
college  faculty  in  the  FTP  were  able  to  design,  develop,  and  implement  very  good  VTT  courses. 
The  community  college  regards  program  flexibility  as  a  necessary  element  in  its  relationship  to 
the  community,  so  this  objective  is  best  analyzed  over  a  long  period  of  time. 


Objective  C3 

Determine  the  Types  of  Staff  Training  and  Technical 
Assistance  Required  By  the  Community  Colleges  to  Develop  and  Implement  a 
Teletraining  Approach  to  Military  Instruction 


The  purpose  of  this  objective  was  to:  (a)  describe  the  types  of  staff  training  that  were 
conducted  for  the  FTP  personnel  who  were  developing  and  delivering  the  VTT  instruction,  and 
(b)  assess  the  effectiveness  of  the  training.  Training  was  conducted  for  five  groups  of 
instructional  staff.  The  administrative  teams  at  1ST  and  FCCJ  attended  some  of  the  training 
sessions  as  dictated  by  their  respective  roles.  The  five  primary  groups  who  received  training 
were: 

•  the  VTT  instructors/course  developers 

•  the  MIAs  who  were  the  SMEs  for  course  design  and  development,  and  the  on- 
camera  military  instructors  for  the  course 

•  two  groups  of  remote  site  facilitators:  the  ICs  and  the  MSCs 

•  technical  personnel  at  both  the  origination  site  and  at  the  remote  sites 

•  graphic  artists  who  produced  the  word  pictures  and  computer  graphics  that  were 
used  during  course  delivery. 

Table  29  provides  an  overview  of  the  types  of  training  provided  to  each  of  the  five  groups 
of  personnel. 

Instruct  tonal  Personnel:  Course  Developers,  MIAs,  ICs,  MSCs 

The  most  extensive  and  intensive  training  was  presented  to  the  VTT  instructors/course 
developers.  None  of  the  community  college  personnel  were  instructional  developers  or  had 
taught  VTT  courses.  In  addition,  they  were  responsible  for  converting  military  courses  from  a 
standard  or  traditional  platform  format  to  VTT.  Therefore,  they  received  a  series  of  training 
workshops  that  varied  in  length  from  several  hours  to  several  days. 

The  first  block  of  instruction  was  presented  by  Dr.  Thomas  E.  Cyrs,  New  Mexico  State 
University.  This  workshop  was  presented  to  the  course  developers  and  to  project  personnel  from 
FCCJ,  1ST,  and  DITRA.  Cyrs’  two-day  workshop,  Essential  Skills  for  Television  Teaching: 
There  is  a  Difference,  focused  on  how  to  design  interactive  learning  strategies,  prepare  on-line 
and  off-line  questions  for  students,  develop  word  pictures,  and  produce  an  interactive  student 
study  guide.  He  also  described  the  components  and  processes  needed  to  modify  courses  for 
television  teaching  (e.g.,  pre-planning,  course  design,  and  staff  training)  and  he  demonstrated  how 
to  present  a  positive  image  on  television. 
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Table  29 

Types  of  Training  Provided  to  FTP  Personnel 


£ 

1  i  .5 

I  it 

^  I' 


£  I 
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Clip  Art 


In  the  second  block  of  instruction,  a  series  of  ISD  workshops  were  presented  to  the  course 
developers.  Workshop  topics  included:  (a)  basic  principles  of  learning  theory  and  instructional 
design,  including  how  to  analyze  and  write  learning  objectives  and  how  to  use  the  conditions  of 
learning  and  events  of  instruction  (Gagne,  1985),  (b)  principles  of  media  selection  and  utilization, 
(c)  instructional  strategies  and  methods  for  VTT,  (d)  an  overview  of  the  TNET  system  and  its 
instructional  capabilities,  and  (e)  an  overview  of  military  training,  including  the  Army's 
requirements  for  Reserve  and  Guard  training,  IDT  vs.  ADT,  and  the  components  of  a  standard 
Army  syllabus.  4 

The  next  block  of  instruction,  TNET  Workshop  I,  was  presented  to  the  technical 
personnel  from  FCCJ  and  the  remote  sites  and  to  the  course  developers/VTT  instructors.  The 
primary  purpose  of  this  workshop  was  to  provide  the  technical  personnel  with  an  in-depth 
orientation  to  the  TNET  system.  Personnel  from  AETD  installed  the  TNET  equipment  at  FCCJ 
prior  to  the  workshop;  this  equipment  was  then  used  for  training  purposes.  While  the  primary 
purpose  of  this  training  was  intended  for  the  technical  personnel,  the  course  developers  also  were 
taught  the  basic  functions  of  the  equipment  and  had  the  opportunity  to  practice  using  TNET. 

A  second  TNET  workshop  presented  by  AETD,  TNET  Workshop  II,  was  presented  to  the 
course  developers  and  the  graphic  artists.  This  workshop  focused  on  the  instructional  rather  than 
the  technical  capabilities  of  the  system.  During  this  workshop,  the  AETD  staff  worked  with  the 
graphic  artists  and  production  team  to  assist  them  in  designing  and  producing  instructionally 
sound  graphics  for  VTT. 

The  final  block  of  formal  staff  training.  Putting  It  All  Together,  was  a  two-day  workshop 
presented  to  all  project  personnel  including  the  course  developers,  MIAs,  ICs,  MSCs,  technical 
personnel  and  the  administrative  POCs  at  each  remote  site.  The  purpose  of  this  workshop  was 
to;  (a)  explain  the  technical  and  instructional  roles  and  responsibilities  of  each  participant,  (b) 
explain  the  specific  instructional  procedures  that  each  participant  should  follow  when 
implementing  VTT  using  TNET,  and  (c)  provide  the  opportunity  for  project  personnel  to  form 
the  instructional  and  technical  teams  that  would  be  necessary  to  implement  VTT  instruction. 
Topics  addressed  in  this  training  included:  a  TNET  orientation,  classroom  protocols,  roles  and 
responsibilities,  contingency  plans,  project  evaluation  requirements  and  responsibilities,  and 
network  policies  and  procedures.  As  a  conclusion  to  this  workshop,  the  instructional  and 
technical  teams  worked  together  to  solve  a  series  of  problems  that  might  arise  during  instruction. 

In  addition  to  the  formal  training,  on-the-job  training  (OJT)  individual  and  team  training 
was  implemented.  This  training  included: 

•  A  13-page  job  aid,  The  Quick  Reference  Guide  to  Course  Conversion,  was 
provided  to  each  course  developer  to  assist  them  in  developing  the  courseware. 
This  job  aid  was  an  abbreviated  handout  of  key  points  and  instructional  ideas  that 
should  be  considered  when  designing  VTT  instruction. 

•  As  the  course  developers/VTT  instructors  and  the  MIAs  developed  each  lesson, 
they  were  given  feedback  and  assistance  in  redesigning  the  lessons. 
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•  The  VTT  instructors  and  MlAs  were  given  instruction  on  television  and 
presentation  techniques  by  personnel  at  FCCJ.  As  part  of  this  instruction,  the 
VTT  instructors  and  MIAs  practiced  a  variety  of  instructional  strategies  and  they 
practiced  teaching  over  the  network. 

•  As  a  component  of  the  OJT,  approximately  ten  sessions  per  course  (for  the  MOS 
courses)  were  conducted  where  all  the  remote  site  coordinators  and  the  technicians 
were  present.  These  practice  sessions  were  provided  to  that  all  personnel  could 
refine  their  roles. 

Technical  Personnel 

The  technical  personnel  received  individual  training  from  the  AETD  staff  on  the  TNET 
system  when  the  equipment  was  installed  at  each  remote  site.  The  technical  personnel  also 
attended  the  Putting  It  All  Together  workshop  and  they  participated  in  the  OJT. 

The  graphic  artists  also  received  specialized  individual  training  from  AETD  on  how  to 
design  graphics  for  TNET.  After  the  training,  the  graphic  artists  conducted  an  extensive 
formative  evaluation  of  the  graphics  and  word  pictures.  This  pilot  testing  was  a  form  of  OJT 
since  there  was  considerable  trial  and  error  to  determine  what  colors,  sizes,  fonts,  and  clip  art 
were  appropriate  for  TNET  and  which  graphics  were  instructionally  sound. 

Evaluations  of  the  Staff  Training:  Quantitative  Data 

Putting  It  All  Together.  The  only  training  that  was  formally  evaluated  by  the  FTP  was 
the  Putting  It  All  Together  workshop.  This  workshop  was  planned  to  insure  that  all  personnel 
involved  in  implementing  the  VTT  instruction  would  work  together  efficiently  and  effectively. 
This  workshop  was  seen  as  a  critical  aspect  of  staff  training.  Participants  were  asked  to  rate 
various  aspects  of  the  workshop  and  it’s  effectiveness  of  the  workshop  in  terms  of  preparing 
them  to  perform  their  project  roles. 

Regarding  aspects  of  the  workshop,  eleven  questions  were  asked  about  the  quality  of  the 
training,  for  example,  the  clarity  of  the  objectives,  the  number  of  opportunities  provided  to  ask 
questions,  the  relevance  of  die  topics,  etc.  Participants  were  asked  to  rate  each  question  on  a  5- 
point  scale.  The  means  ranged  4.14  to  4.95,  indicating  that  the  participants  thought  that  the 
workshop  was  very  good  to  excellent. 

Participants  were  also  asked  about  the  adequacy  of  the  training  to  prepare  them  to  perform 
their  roles  and  responsibilities  for  the  FTP.  Table  30  summarizes  the  results.  Of  the  ten 
questions  asked,  eight  had  means  of  4.00  or  higher,  indicating  that  the  participants  felt  they  were 
very  well  prepared  to  perform  most  of  the  roles  and  tasks  that  they  would  be  asked  to  perform 
while  implementing  the  VTT  instruction.  The  lowest  mean,  3.41,  was  obtained  when  participants 
were  asked  about  how  well  prepared  they  felt  to  operate  the  TNET  equipment. 
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Table  30 


Participant  Evaluation  of  the  Putting  It  All  Together  Workshop 


1  Relative  to  the  specific  objectives  planned  for  the  workshop,  how  well  prepared  do  you  feel  that  you  now  are  1 
1  for  vour  roles  on  future  protect  tasks?  Circle  one  answer  for  each  item.  1 

Extremely  Very 

Well  Poorly 

Prepared  Prepared 

i  1 

5  4  3  2  l  N/A 

N  s  22 

X(SD) 

%  4&5 

a.  Understand  instructional  roles  and 
responsibilities  for  course  delivery. 

4.20(0.77) 

72 

b.  Understand  managerial/organizational 

roles  and  responsibilities  for  course  delivery. 

4.45(0.60) 

95 

c.  Perform  basic  TNET  operations. 

3.41(0.45) 

45 

d.  Implement  contingency  plans  during 
course  delivery. 

4.05(0.72) 

77 

e.  Use  the  prescribed  method  to  provide 
feedback  (After  Action  Review)  on  a 
daily  basis  to  the  origination  site. 

4.00(0.88) 

63 

f.  Implement  classroom  instructional 

protocol  during  course  delivery. 

4.42(0.61) 

82 

g.  Understand  instructional  strategies 
that  will  be  used  via  TNET. 

4.38(0.67) 

96 

h.  Apply  designated  policies  and 
procedures  at  the  respective 
community  colleges  during  course  delivery. 

4.50(0.61) 

86 

1  L  Implement  data  collection  and  evaluation 

procedures  during  course  delivery. 

3.90(0.97) 

63 

j.  Follow  prescribed  procedures  for 

make-up  and  remediation  instruction 
during  course  delivery. 

■ 

4.15(0.59) 

82 

Participants  were  also  asked  open-ended  questions  about  the  aspects  of  the  workshop  that 
they  felt  were  particularly  effective,  what  needed  improvement,  and  what  kinds  of  follow-up 
training  was  needed.  Two  consistent  themes  emerged  for  each  of  the  three  questions.  First, 
hands-on  experience  with  the  TNET  equipment  and  how  to  use  TNET  were  the  aspects  of  the 
workshop  that  were  listed  most  often  as  particularly  effective.  They  were  also  listed  as  the 
aspects  that  needed  to  be  addressed  in  follow-up  workshops.  Second,  the  interaction  that  took 
place  between  the  civilian  and  military  personnel  during  the  workshop  was  listed  as  both 
necessary  and  effective;  this  interaction  was  also  listed  as  an  aspect  of  the  workshop  that  should 
be  addressed  again  in  future  workshops. 

Sample  comments  about  what  was  effective  included: 

•  "The  interaction  between  the  military  and  civilians.  It  is  two  different  schools  of 
thought  and  it  was  essential  that  the  military  present  it’s  (sic)  needs.  I  think 
mutual  respect  was  achieved  on  both  sides." 

•  "Meeting  people  from  the  remote  sites,  time  to  practice  on  TV,  one  more  of  this 
500  piece  puzzle  has  fallen  into  place." 

•  "Hands-on  equipment,  good  to  problem  solve  with  the  military." 

Sample  comments  about  what  was  needed  for  future  workshops  included: 

•  "More  instruction  regarding  technical  functions." 

•  "Maybe  a  little  more  role-playing  in  use  of  equipment  and  maybe  a  view  of 
military  tegs  and  POI.  Overall  the  global  presentation  went  well.” 

•  "Would  have  liked  to  spend  some  time  in  a  group  with  military/civilian  personnel 
involved  in  special  course  I  am  facilitating." 

•  "More  site  interaction." 

On-the-job  Training.  In  addition  to  the  data  obtained  from  the  Putting  It  All  Together 
workshop,  a  formal  feedback  system  was  established  to  provide  information  to  the  VTT 
instructors  (for  the  MOS  courses)  and  to  the  network  administrators  at  FCCJ  about  the 
instructional,  technical,  and  administrative  aspects  of  their  practice  presentations.  Each  VTT 
instructional  practice  session  was  rated  on  a  scale  of  1  to  10  by  all  personnel  attending  the 
practice  sessions  at  the  remote  sites  (the  ICs,  MSCs,  and  any  attending  personnel  from  1ST  or 
DITRA).  These  ratings  were  used  to  provide  feedback  about  the  instructional  and  technical 
adequacy  of  the  instruction.  Corrective  measures  were  devised  as  needed  based  on  the  data.  FTP 
personnel  expected  that  the  feedback  sheets  would  show  improvements  from  the  first  sessions 
to  the  last;  however,  there  were  too  many  variations  in  the  practice  sessions  to  obtain  reliable 
results. 
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Evaluations  of  the  Staff  Training:  Narrative  Data 

In  addition  to  the  data  presented  above  and  as  part  of  the  overall  project  evaluation,  the 
instructional  personnel  were  asked  what  types  of  training  they  received  and  how  effective  the 
training  was.  The  narrative  data  are  consistent  with  data  discussed  above. 

Essential  Skills  for  Television  Teaching.  The  FTP  did  not  conduct  a  formal  evaluation 
of  Dr.  Cyrs’  workshop.  However,  the  following  are  representative  comments  about  Cyrs’ 
presentation  collected  as  part  of  the  project  evaluation: 

•  "Cyrs*  workshop  provided  me  with  various  options  to  use  in  choosing  instructional 
techniques  for  the  course." 

•  "I  used  word  pictures  as  a  basic  outline  format,  eschewing  the  more  elaborate  and 
visually  striking  designs  suggested  by  Cyrs.” 

The  most  concrete  evaluation  of  the  effectiveness  of  Cyrs’  training  was  the  fact  that  word 
pictures  (Cyrs  &  Smith,  1990)  were  adopted  as  the  primary  means  of  displaying  visuals  during 
instruction  and  as  the  primary  strategy  used  to  create  an  ISG.  Approximately  1000  visuals/word 
pictures  were  developed  for  each  of  the  MOS  courses  and  approximately  200  were  developed  for 
each  of  the  special  topics  courses.  Adoption  of  this  strategy  with  few  modifications  attests  to 
the  value  of  Cyrs’  workshop. 

Workshops  on  Instructional  Design,  Selecting  and  Using  Media  and  Instructional 
Strategies,  and  Overview  of  Military  Training.  Like  the  Cyrs’  workshop,  the  workshops 
presented  by  1ST  were  not  formally  evaluated.  However,  the  community  college  personnel  had 
no  formal  training  in  ISD  and  only  one  had  conducted  any  military  training.  Their  knowledge 
of  ISD  and  military  training  was  essential  for  them  to  convert  the  instruction  from  platform  to 
VTT  instruction.  Representative  comments  from  the  VTT  instructors  about  the  adequacy  of  this 
training  included: 

•  "Guidelines  received  from  [1ST  personnel]  played  a  major  role  in  determining  my 
instructional  techniques." 

•  "I  followed  the  principles  of  active  learning  as  espoused  in  the  workshops 
conducted  by  [1ST  personnel]  and  Cyrs." 

•  "The  hounding  voice  of  [1ST  personnel]  was  the  primary  incentive  for  selecting 
and  using  different  types  of  interactivity  during  the  course." 

TNET  Workshops  I  and  n.  No  formal  evaluation  of  the  two  workshops  presented  by 
AETD  was  conducted.  However,  the  instructional  personnel  were  consistent  in  making  comments 
throughout  staff  training  and  course  delivery  that  they  needed  to  feel  more  comfortable  with  the 
TNET  equipment.  They  also  indicated  they  needed  more  time  to  practice  using  TNET  even 
though  there  was  a  trained  technician  at  each  site  when  TNET  was  being  used.  The  technical 
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personnel  did  not  make  any  comments,  positive  or  negative,  about  the  need  for  further  training 
on  the  TNET  equipment.  These  comments  from  the  instructional  staff  do  not  necessarily  reflect 
any  inadequacies  in  the  AETD  training.  While  the  equipment  is  complex,  it  is  not  difficult  to 
use.  However,  the  comments  about  TNET  from  the  instructional  personnel  do  point  to  the  need 
to  provide  considerable  practice  with  any  new  technology. 

Some  representative  comments  from  FTP  instructional  personnel  include: 

•  "The  difficulty  of  this  medium  is  a  matter  of  practice." 

•  "Need  to  be  familiar  with  the  equipment,  only  taught  two  hours  on  TNET." 

•  "[More]  formal  on  network  training  program;  had  fear  of  crashing  the  network." 

•  "Just  more  practice  time  with  the  technology." 

•  "More  of  the  same  types  of  training  [practice  sessions  over  TNET.  Cyrs 
workshop]." 

•  "Probably  more  TV  training." 

•  "Need  more  hands  on  for  all  IC,  POC.  &  technical." 

•  "Needed  more  training  on  how  to  run  computer  (was  totally  dependent  on 
technician). 

In  summary,  staff  training  is  an  essential  aspect  of  presenting  VTT  instruction.  The 
course  developers/VTT  instructors  received  the  most  extensive  training  running  the  gamut  from 
how  to  reconfigure  military  courses  for  VTT  delivery  to  technical  training  about  how  to  use 
TNET.  While  they  received  considerable  formal  training  and  OJT  about  all  aspects  of 
implementing  teletraining,  they  stated  that  they  needed  more  training  on,  and  practice  with,  the 
technology.  Other  non-technical  personnel  (ICs  and  MSCs)  also  indicated  that  they  wanted  more 
TNET  training. 

The  Putting  It  All  Together  workshop  was  designed  to  help  all  instructional  and  technical 
personnel  function  effectively  in  the  their  respective  roles  and  work  together  as  a  team.  It  was 
very  effective,  as  indicated  by  means  above  4.0  on  all  but  two  response  categories.  While  not 
formally  evaluated,  the  narrative  responses  indicated  that  the  knowledge  and  skills  gained  from 
the  other  workshops,  e.g.,  those  presented  by  Cyrs  and  1ST  personnel,  were  important  and 
necessary  for  designing  and  conducting  VTT  using  TNET. 
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Objective  C4 

Determine  the  Fit  Between  Selected  Military  Training  Needs 
and  the  Community  College  Curriculum 


The  purpose  of  this  objective  was  to  determine  the  areas  of  possible  overlap  between 
military  training  needs  and  the  existing  community  college  curriculum.  While  little  project  data 
was  collected  on  this  objective,  some  related  research  exists  in  the  literature.  Based  on  this 
research,  a  conceptual  analysis  of  the  military  training  needs  and  the  community  college 
curriculum  was  conducted. 

Military  Training  Needs 

The  military  divides  its  training  needs  into  two  broad  categories:  individual  training 
(tasks  that  must  be  performed  by  a  single  person)  and  collective  training  (training  for  tasks  that 
must  be  performed  by  groups  of  individuals).  Collective  tasks  are  often  categorized  by  the  size 
of  the  group  that  must  perform  them  (e.g.,  squad,  platoon,  company,  etc.),  whereas  individual 
tasks  are  categorized  by  who  performs  the  tasks  (e.g.,  leader  tasks,  soldier  tasks  and  those  tasks 
that  are  common  to  leaders  and  soldiers)  (TRADOC  1990). 

The  military  uses  FTXs,  situational  training  exercises  (STXs),  battle  drills  and  constructive 
simulations  to  train  collective  tasks.  Collective  tasks  normally  involve  actions  that  are  performed 
during  actual  combat  and  are  normally  trained  exclusively  by  the  military.  The  military 
traditionally  uses  its  system  of  formal  schools  to  train  individual  tasks.  Since  many  individual 
tasks  require  the  use  of  non-combat  skills,  the  military  has  sometimes  used  civilian  training 
institutions  to  provide  individual  training. 

The  military  also  recognizes  special  issues  related  to  individual  training  in  the  RC.  These 
special  RC  needs  generally  fall  into  one  of  three  major  areas:  (a)  job  skill  enhancement  (i.e., 
computer  operator),  (b)  regular  academic  or  technical  programs  directly  corresponding  to  a 
military  skill  (i.e..  Licensed  Practical  Nurse),  and  (c)  specific  military  skills  that  civilian 
providers,  such  as  community  colleges,  have  the  resources  to  design  and  develop  as  special 
offerings  (i.e.,  military  surveying),  (Watt,  1988). 

Existing  Civilian/Military  Programs 

The  military  services  have  established  several  avenues  that  can  be  used  to  obtain  military 
training  from  civilian  institutions.  The  Navy’s  CIVTRAIN  program  is  a  database  that  lists 
colleges  and  technical  institutes  that  can  be  used  to  provide  selected  occupational  training  to 
reservists  in  a  limited  geographical  area. 

The  Army’s  Forces  Command/T raining  and  Doctrine  Command  (FORSCOM/TRADOC) 
Regulation  135-3  contains  a  list  of  occupational  specialty  skills  that  are  considered  appropriate 
for  civilians  to  train  under  contract  to  the  Army  (contract  training)  (Watt,  1988).  These  53  MOSs 
are  listed  in  Table  31. 
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Table  31 

Extract  FORSCOM/TRADOC  Regulation  135-3* 


,  *<  i  vt  -I*. 

’  V*.  /  <  ?  f 


Tactical  Microwave  Systems  Repairer 
Tactical  Satellite/Microwave  Systems  Operator 
Radio/Television  Systems  Specialist 
Field  Radio  Repairer 
Teletypewriter  Repairer 

Tactical  Communications  System  Operator/Mechanic 

Electronic  Instrument  Repairer 

Special  Electrical  Devices  Repairer 

Calibration  Specialist 

Avionics  Mechanic 

Cable  Splicer 

Topographic  Instrument  Repair  Specialist 
Audiovisual  Equipment  Repairer 
Office  Machine  Repairer 
Reproduction  Equipment  Repair  Specialist 
Metal  Worker 
Machinist 

Carpentry  and  Masonry  Specialist 
Structures  Specialist 
Materials  Quality  Specialist 
Firefighter 

Water  Treatment  and  Plumbing  Systems  Specialist 
Electrician 

Utilities  Equipment  Repairer 
Prime  Power  Production  Specialist 
Industrial  Gas  Production  Specialist 
Quarry  Specialist 

Concrete  and  Asphalt  Equipment  Operator 

Aircraft  Power  Plant  Repairer 

Aircraft  Electrician 

Aircraft  Structural  Repairer 

Aircraft  Pneudraulic  Repairer 

Court  Reporter 

Broadcast  Journalist 

Card  and  Tape  Writer 

Computer/Machine  Operator 

Technical  Drafting  Specialist 

Cartographer 

Illustrator 

Construction  Surveyor 
Topographic  Surveyor 
Photo  and  Layout  Specialist 
Photolithographer 
Still  Photographic  Specialist 
Motion  Picture  Specialist 
Audio/Television  Specialist 
Practical  Nurse 

Medical  Laboratory  Specialist  (Basic  &  Advanced) 

Petroleum  Lab  Specialist 

Chemical  Lab  Specialist 

Meteorological  Observer 

Field  Artillery  Meteorological  Crewman 

Food  Services  Specialist 
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The  53  MOSs  approved  for  contract  training  were  compared  to  the  course  and  program 
descriptions  listed  in  the  course  catalogs  from  the  three  community  colleges  that  participated  in 
the  FTP.  It  was  determined  from  this  comparison  that  16  of  these  MOSs  have  some  relationship 
to  courses  or  programs  currently  listed  in  the  community  colleges’  catalogs.  Table  32  lists  these 
16  MOSs  and  classifies  the  instruction  available  at  the  community  college  as  either  a  course(s) 
or  complete  associate  degree  program(s). 

Although  the  table  lists  the  programs  and  courses  where  there  is  a  similarity  of  content, 
there  was  no  in-depth  content  analysis  to  determine  the  exactness  of  the  match  between  the 
military  and  community  college  content.  It  is  unclear  whether  the  content  of  the  community 
college  programs  and  courses  is  close  enough  to  provide  certification  in  the  Army  MOSs.  This 
table  shows  that  a  course  or  program  of  this  type  is  available  from  one  of  the  community 
colleges  who  participated  in  this  project,  but  it  does  not  specify  which  one.  Objective  C5 
addresses  which  community  college  courses  are  available  with  content  similar  to  the  VTT  course 
taught  in  the  FTP. 

Table  32 

Relationship  Between  TRADOC  Approved  Contract  Training  and  FTP  Community  College  Curricula 


26T  Radio/Television  Systems  Specialist 

Yes 

Yes 

35B  Electronic  Instrument  Repairer 

Yes 

Yes 

35K  Avionics  Mechanic 

Yes 

No 

44B  Metal  Worker 

Yes 

No 

3  IB  Carpentry  and  Masonry  Specialist 

Yes 

Yes 

51G  Materials  Quality  Specialist 

Yes 

No 

SIM  Firefighter 

Yes 

Yes 

3  IN  Water  Treatment  and  Plumbing  Systems  Specialist 

Yes 

No 

SIR  Electrician 

Yes 

No 

68G  Aircraft  Structural  Repairer 

Yes 

No 

74D  Computer/Machine  Operator 

Yes 

Yes 

81B  Technical  Drafting  Specialist 

Yes 

Yes 

83E  Photo  and  Layout  Specialist 

Yes 

No 

91C  Practical  Nurse 

Yes 

Yes 

92B  Medical  Laboratory  Specialist 

Yes 

Yes 

.94B  Food  Service  Specialist 

Yes 

No 
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In  summary,  only  individual  rather  than  collective  tasks  can  be  trained  by  civilian 
institutions.  Of  the  S3  MOSs  listed  as  appropriate  for  civilian  contract  training  by 
FORSCOM/TRADOC  Regulation  135-3,  only  16  of  these  are  available  as  courses  or  programs 
at  one  or  more  of  the  three  Florida  community  colleges  that  participated  in  this  study. 
Interestingly,  none  of  the  MOSs  taught  during  the  FTP  are  listed  in  FORSCOM/TRADOC 
Regulation  135-3,  and  therefore  these  MOS  courses  (i.e.,  71L10,  76Y10,  95B10)  were  not 
addressed  in  the  comparison  table  (Table  32).  However,  extensive  criminal  justice  programs  exist 
at  both  FCCJ  and  SPJC.  Basic  Military  Police  (95B10)  was  selected  to  be  taught  during  this 
project  in  part  because  of  existence  of  these  programs. 
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Objective  C5 

Determine  the  Match  Between  Selected  Military  Courses 
and  Standard  Community  College  Offerings  and  if  Completion 
of  Military  Courses  Can  Result  in  Community  College  Credit 

The  purpose  of  this  objective  was  to  determine  the  fit  between  the  military  courses 
selected  for  the  pilot  test  and  the  civilian  courses  found  in  the  curriculum  at  the  three  community 
colleges.  Community  college  administrative  personnel  who  originally  proposed  the  project 
considered  this  fit  very  important  in  encouraging  student  involvement,  motivating  students  to 
achieve,  and  giving  the  classroom  a  college  atmosphere.  This  objective  also  includes  an 
investigation  of  two  issues  surrounding  the  award  of  college  credit:  (a)  whether  credit  can  be 
given  for  military  courses  taken  at  a  community  college,  and  (b)  the  importance  of  college  credit 
to  students. 

Although  the  MOS  courses  chosen  for  the  FTP  are  not  contained  in  the  list  of  courses 
approved  for  contract  training  found  in  FORSC OM/TRADOC  Regulation  135-3  (see  Objective 
C4),  the  courses  were  selected  by  DITRA  based  on  the  recommendations  of  the  FLARNG  and 
the  81st  ARCOM.  The  FLARNG  and  the  81st  ARCOM  selected  the  courses  partly  based  on  the 
expected  throughput  for  the  training  year.  The  courses  were  selected  by  DITRA  and  later 
analyzed  by  the  FTP  and  were  appropriate  for  VTT  (see  Objective  A4). 

Course  Content  Analysis 

Initially,  the  military  and  community  college  courses  were  compared  to  determine  the 
extent  of  overlap  between  them.  These  comparisons  were  made  by  analyzing  course  descriptions 
found  in  the  three  community  college  catalogs  and  comparing  them  to  the  content  presented  in 
the  five  VTT  courses.  For  example,  in  the  71L10  course,  each  college  has  an  introductory  course 
in  keyboarding  or  beginning  typewriting  that  could  contain  content  that  is  also  presented  in  the 
military  VTT  course.  In  contrast,  the  TQL  course  lacked  overlap  with  any  course  offered  by  the 
three  community  colleges.  This  comparison  formed  the  basis  for  determining  whether  or  not 
college  credit  could  be  given  to  students  for  the  five  military  courses. 

Community  College  Credit 

The  Guide  to  the  Evaluation  of  Educational  Experiences  in  the  Armed  Services  (American 
Council  on  Education,  1988)  is  a  standard  reference  work  for  establishing  credit  for  learning 
acquired  in  the  military.  The  Guide  recommends  both  college  and  vocational  credit.  However, 
these  recommendations  are  based  on  both  classroom  and  experiential  learning.  Therefore, 
certification  in  skill  level  10  is  only  part  of  the  knowledge  that  a  service  member  will  gain  during 
his  or  her  performance  of  their  occupational  duties.  This  means  that  the  material  covered  in  the 
VTT  versions  of  the  three  Army  MOS  courses  does  not  reflect  the  full  scope  of  experiential 
learning  upon  which  The  Guide’s  recommendation  is  based.  For  example,  76Y10  and  95B10 
include  both  IDT  and  ADT  phases  and  the  VTT  courses  addressed  only  the  IDT  phase. 


113 


The  1988  Guide  recommends  awarding  the  following  college  credit  for  the  three  Army 
MOS  courses: 

71L10:  Administrative  Specialist 

Vocational  certificate  or  lower-division  baccalaureate/associate  degree  category 

•  2  semester  hours  in  typing 

•  2  semester  hours  in  recordkeeping 

•  1  semester  hour  in  business  communications 

•  3  semester  hours  in  office  procedures 

76Y10:  Unit  Supply  Specialist 

Vocational  certificate  or  lower-division  baccalaureate/associate  degree  category 

•  3  semester  hours  in  recordkeeping 

•  3  semester  hours  in  office  procedures 

•  2  semester  hours  in  office  machines 

•  1  semester  hour  in  typing 

95B10:  Basic  Military  Police 

Lower-division  baccalauieate/associate  degree  category 

•  3  semester  hours  in  patrol  operations 

Determining  Credit  for  FTP  Courses.  The  Instructional  Manger  at  FCCJ  coordinated 
the  efforts  of  the  community  college  network  to  provide  college  credit  for  the  military  courses 
offered  in  the  project.  The  community  colleges  faced  several  difficult  challenges  in  their  attempt 
to  standardize  the  credit  issued  by  each  college  for  the  respective  military  courses. 

In  the  state  of  Florida,  each  community  college  is  governed  by  a  separate  board  of 
trustees.  This  allows  each  college  the  freedom  to  create  an  organizational  structure  and  develop 
a  curriculum  that  is  most  responsive  to  the  community  it  serves.  Each  community  college  in  the 
state  of  Florida  follows  a  common  course  numbering  system.  This  system  provides  a  degree  of 
curriculum  standardization  among  the  state  supported  community  colleges.  However,  due  to  the 
differing  organizational  structures  and  variations  in  curricula,  the  same  college  credit  was  not 
awarded  at  each  of  the  community  college  sites. 

There  were  no  equivalent  college  courses  in  the  state  common  course  numbering  system 
for  76Y10,  HazWaste,  and  TQL.  In  addition,  instructional  content  for  the  military  courses 
selected  for  the  project  usually  contained  only  partial  content  from  several  community  college 
courses  and  did  not  fully  cover  the  content  in  any  single  course.  In  76Y10,  for  example,  the 
topics  addressed  did  not  sufficiently  cover  enough  standard  community  college  course  content 
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to  award  college  credit.  In  these  cases  the  students  were  given  non-degree  earning  special  topics 
and  continuing  education  credits  (CEC). 


In  71L10,  it  was  possible  to  substitute  military  correspondence  style  and  production  typing 
for  other  topics  normally  covered  in  the  standard  curriculum  and  award  college  credit.  College 
credit  was  awarded  in  the  95B10  course  along  with  additional  CECs  that  were  useful  to  RC 
students  working  in  the  security  and  law  enforcement  fields.  Table  33  shows  the  type  and 
amount  of  credit  granted  by  each  participating  community  college  for  each  course.  As  shown 
in  the  table,  students  participating  in  the  project  at  different  institutions  received  different 
amounts  of  college  credit. 

Table  33 

College  Credit  Awarded  for  Military  VTT  Courses 


it*  ■  ?  "  ;■  ■ 

71L10 

OST  1 100,  Keyboarding 
(3  College  Credits) 

OST  1100  Keyboarding  I 
(3  College  Credits) 

OST  1100  Keyboarding  I 
(3  College  Credits) 

. 

BED  0570  Administrative 

Specialist  (non-credit) 

OST  1321  Automated  Office 
Skills  (2  College  Credits) 

BUS  400  Administrative 

Specialist  Training  (non-credit) 

999  Office  Systems 
Technology:  Selected 

Topics  (non-credit) 

I  76Y10 

BED  0650  Unit  Supply 

Specialists  (non-credit) 

BUS  0448,  Record  Keeping 
(non-credit) 

OST  1321  Automated  Office 
Skills  (non-credit) 

443  Career  Skills 
Development:  Selected 
Topics  (non-credit) 

95B10 

CJT  2930  Special 

Topics/Setninars  in  Criminology 
(3  College  Credits) 

CCJ  2990  Introduction  to 

Criminal  Justice  for  Military 
Police  (3  College  Credits) 

CCJ  2930  Criminal 

Justice:  Selected  Topics  (3 
College  Credits) 

PSR  0200  Basic  Law 

Enforcement  (non-credit) 

PSR  0242  Military  Police  (non¬ 
credit) 

FLE  1781  Introduction  to 

Criminal  Justice  for  Military 
Police  (non-credit) 

805  Police  Tactical 

Response  (non-credit) 

HAZ 

PSR  0840  Handling  Hazardous 
Waste  (non-credit) 

TEL  698  Hazardous  Materials 
and  Waste  Management 
(non-credit) 

918,  Hazardous  Waste 
(non-credit) 

I  TQL 

BED  0071  Introduction  to 

Quality  Leadership  (non-credit) 

BUS  466  Total  Quality 

Leadership  (non-credit) 

674  Management: 

Selected  Topics  (non¬ 
credit) 
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Student  Responses 


While  it  was  advantageous  for  the  community  colleges  to  award  college  credit,  and  it  was 
assumed  that  students  would  want  college  credit,  students  in  the  three  Army  MOS  courses  were 
asked  how  important  the  college  credit  was  to  them.  They  were  asked  to  respond  on  a  3-point 
scale  from  very  important  (3)  to  not  important  (1).  The  students  tended  to  respond  differently 
depending  upon  whether  they  were  enrolled  or  planning  to  enroll  in  a  college  program.  Student 
responses  are  shown  in  Table  34  and  were  categorized  by  whether  students  were  enrolled  in 
college  or  not.  (Students  in  the  HazWaste  and  TQL  courses  were  not  asked  this  question). 

Table  34 

Importance  of  College  Credit  to  Students 


71L10 

1.94  (.83) 

1.96  (.82) 

1.90  (.88) 

33 

23 

10 

76Y10 

1.64  (.87) 

1.74  (.90) 

1.00  (.00) 

40 

35 

5 

95B10 

2.12  (.77) 

2.17  (.78) 

1.67  (.58) 

27 

24 

As  shown  in  the  table,  the  students  enrolled  in  or  planning  to  enroll  in  college  programs 
tended  to  rate  the  importance  of  college  credit  somewhat  higher  than  those  students  who  were 
not  planning  to  continue_their  education.  With  the  exception  of  the  76Y10  students  who  were 
not  enrolled  in  college  (X  =  1.00),  all  of  the  other  students  stated  that  college  credit  was  at  least 
moderately  important  to  them. 

In  summary,  there  were  many  areas  where  there  was  some  content  overlap  between  the 
military  courses  selected  for  the  FTP  and  the  standard  course  offerings  of  the  three  community 
colleges  involved  in  the  project.  In  areas  where  the  content  was  heavily  military,  there  was  not 
sufficient  overlap  to  award  standard  community  college  credit.  In  these  cases,  the  community 
colleges  awarded  CECs  or  a  mix  of  CECs  and  degree  earning  credit.  The  same  college  credit 
was  not  uniformly  awarded  across  the  community  colleges.  In  short,  the  match  between  the 
military  courses  and  the  standard  community  college  course  offerings  varied  among  courses  and 
across  institutions,  but  it  was  possible  to  award  some  form  of  college  credit,  either  standard 
community  college  or  continuing  education,  for  completion  of  the  military  courses.  Military 
students,  especially  those  already  enrolled  in  college  programs,  typically  want  college  credit 
awarded  whenever  it  is  possible. 
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Objective  C6 

Determine  the  Community  Colleges’  Level  of  Cooperation 
ami  Support  for  Delivery  of  VTT  Courses,  and  Discuss  Their 
Future  Effectiveness  in  Providing  a  VTT  Approach  to  Military  Instruction 


One  of  the  primary  goals  of  the  FTP  was  to  determine  the  feasibility  of  community 
colleges  offering  military  courses  by  teletraining.  Support  from  and  among  the  community 
colleges  was  critical  to  the  success  of  this  project,  and  likewise,  to  any  future  venture  between 
the  military  and  community  colleges.  This  objective  examines  the  level  of  community  college 
cooperation  required  to  successfully  design,  develop,  and  deliver  effective  VTT  instruction. 

Contractual  Relationships 

SPJC  and  VCC  were  selected  as  remote  TNET  sites  because  they  were  located  in  regions 
containing  a  large  number  of  the  desired  training  population  at  the  time  of  course  selection.  They 
then  became  subcontractors  to  FCCJ  in  order  to  accomplish  this  pilot  study.  Under  the  terms  of 
these  contracts,  SPJC  and  VCC  agreed  to  provide  the  following: 

•  access  to,  and  use  of  a  designated  classroom  capable  of  housing  the  TNET 
equipment  and  15  students 

•  staff  support  and  instructional  management  services 

•  a  content  qualified  IC  to  distribute  and  collect  course  materials,  to  oversee 
and  monitor  evaluations  and  tests,  and  to  collect  project  evaluation  data 

•  an  instructional  management  supervisor/POC  to  handle  administrative 
requirements 

•  technical  personnel  to  perform  the  minimal  equipment  maintenance 

•  ancillary  resources,  equipment,  laboratories,  and  all  other  required  facilities, 
including  typewriters,  VCR’s,  and  microfiche  readers 

•  student  services  as  customarily  covered  by  registration  fees. 

The  interactions  between  the  community  colleges  were  effective  due  in  part  to 
comprehensiveness  of  the  subcontracts  and  to  the  staff  training  that  was  conducted  (see  Objective 
C3).  A  specific  training  workshop  was  held  with  the  explicit  purpose  of  insuring  that  all 
personnel:  (a)  knew  their  roles  and  responsibilities  and  (b)  had  the  chance  to  get  acquainted  prior 
to  delivery  of  the  instruction.  In  addition,  practice  sessions,  designed  for  the  same  purposes, 
were  conducted  over  the  network  for  instructional  and  technical  support  personnel. 
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Administrative  Personnel 


A  number  of  community  college  personnel  were  needed  to  facilitate  the  interaction  between 
community  colleges  and  to  participate  in  the  design,  development  and  delivery  of  the  courses. 
A  logistics  manager  at  FCCJ  was  responsible  for  monitoring  the  contracts  between  the 
community  colleges.  The  task  of  formulating  and  administering  the  contracts  was  time 
consuming,  but  essential.  The  logistics  manager  communicated  with  administrative  POCs  located 
at  SPJC  and  VCC.  The  POCs  were  responsible  for  overseeing  and  coordinating  the  project 
activities  at  each  community  college  in  agreement  with  the  FCCJ  subcontract.  The  out-of-state 
sites,  that  were  not  at  community  colleges,  did  not  have  this  contact  and  were  not  as  well 
prepared  and  organized  for  course  delivery. 

FCCJ  stated  that  it  was  very  important  to  the  success  of  the  project  to  have  top  level 
administrative  support,  i.e.,  from  the  Presidents  of  the  respective  institutions.  These 
administrators  of  the  Florida  community  colleges  meet  together  regularly.  Their  knowledge  of 
and  support  for  the  project  was  a  factor  in  its  success. 

Course  Credit 

One  of  the  most  challenging  administrative  problems  was  locating  individuals  on  the 
campus  with  the  authority  to  create  a  means  of  awarding  college  credit  for  the  military  training 
course.  Despite  the  variety  of  college  course  offerings,  creating  an  appropriate  credit  for  the 
VTT  course  was  difficult,  as  discussed  in  Objective  CS.  In  some  cases,  administrators  found 
little  or  no  connection  with  existing  courses  or  academic  departments.  Credit  arrangements  with 
the  community  colleges  were  made  by  the  Instructional  Manager  at  the  origination  site. 

Each  college  did  provide  the  services  and  benefits  expected  by  tuition-paying  students. 
For  example,  students  had  to  register  for  the  classes,  and  class  and  student  records  were 
maintained  by  the  colleges.  Students  were  also  provided  access  to  academic  advisement  and 
remediation. 

Financial  Detalk 

It  was  important  for  the  community  colleges  who  participated  in  the  FTP  to  pay  their  own 
expenses  and  break  even  when  the  project  was  completed.  In  exchange  for  providing  special 
services  for  the  FTP,  the  community  colleges  charged  a  contractually  fixed  fee  per  course  that 
was  paid  out  of  project  funding.  This  charge  included  room,  support  facilities,  and  services.  In 
addition,  the  tuition  charge  for  each  student  was  paid  out  of  project  funds. 

In  examining  the  benefits  to  the  community  colleges,  the  remote  sites  received  money  that 
covered  the  salaries  of  the  ICs  as  well  as  any  support  staff  needed  to  complete  this  project.  This 
funding  was  in  addition  to  the  tuition  costs  and  costs  for  materials.  The  instructional  staff  also 
received  training  from  FTP  personnel  on  how  to  conduct  VTT  and  how  to  use  the  VTT 
equipment.  Substitute  teachers  were  also  hired  out  of  project  funds  to  replace  the  teachers 
needed  as  instructors  at  the  origination  site. 
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In  conclusion,  community  colleges  have  a  history  of  working  together.  If  VTT  is 
considered  a  viable  option  to  meet  certain  DoD  training  needs,  the  community  colleges  that 
participated  in  the  FTP  have  demonstrated  that  they  can  and  will  establish  networks  to  deliver 
instruction  via  distance  learning.  While  administratively  difficult,  the  community  colleges  in  the 
FTP  were  willing  and  able  to  grant  college  credit  for  military  courses. 
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Objective  C7 

Identify  Key  Organizational/Management  Issues  Facing 
Community  Colleges  in  Implementing  a  VTT-Based  Approach  to 

Military  Instruction 

The  purpose  of  this  objective  was  to  determine  the  organizational  and  managerial  factors 
that  influenced  how  successful  the  community  colleges  were  in  implementing  military  courses 
using  TNET.  There  is  no  specific  data  that  can  be  reported  for  this  objective.  Rather,  the  issues 
identified  by  the  project  management  teams  at  FCCJ  and  1ST  were  the  primary  inputs  for  this 
objective.  The  major  issues  identified  were:  contractor/subcontractor  relationships,  project 
evaluation,  and  community  college  and  military  coordination. 

Contractor/Subcontractor  Relationships 

The  contractor/subcontractor  relationships  in  this  project  were  multi-layered.  DITRA 
contracted  with  1ST  to  serve  as  the  prime  contractor  for  the  project  and  to  provide  technical, 
administrative,  and  managerial  assistance  to  FCCJ  who  was  the  anchor  community  college  and 
prime  subcontractor.  FCCJ,  in  turn,  had  two  primary  responsibilities:  (a)  to  coordinate  the  efforts 
of  the  remote  sites  with  the  two  other  community  colleges  and  (b)  to  design,  develop,  and 
implement  the  VTT  courses. 

The  government  agency  originally  responsible  for  conducting  the  FTP  was  TPDC.  This 
office,  located  in  the  same  city  as  the  prime  contractor,  was  closed  two-thirds  of  the  way  through 
the  project.  DITRA  became  the  government  agency  responsible  for  overseeing  completion  of  the 
project.  While  this  change  did  not  cause  undue  hardship,  having  TPDC  in  the  same  city  as  the 
prime  contractor  was  convenient  for  coordinating  project-related  efforts  and  having  face-to-face 
meetings.  In  addition,  the  change  of  government  agencies  meant  that  the  new  agency  initially 
had  a  limited  context  for  the  project. 

The  contractor  and  subcontractors  were  also  located  in  different  cities.  While  distance 
was  a  relatively  minor  issue  in  these  relationships,  face-to-face  meetings  between  the  project 
teams  at  the  university  and  community  college  and  among  the  community  colleges  required 
travel. 


The  relationships  among  the  three  community  colleges  that  participated  in  this  project 
were  established  through  subcontracts.  These  relationships  were  previously  discussed  in 
Objective  C6  and  are  not  repeated  here.  Some  of  the  key  issues  that  the  community  colleges  had 
to  contend  with  were  identifying  and  hiring  personnel,  assigning  college  credit,  and  coordinating 
project  activities. 

Some  aspects  of  the  contractual  relationships  that  should  be  considered  when  conducting 
a  large  project  are: 

•  It  is  important  for  any  organization  participating  in  a  project  such  as  the  FTP  to 
obtain  information  in  a  timely  fashion.  In  large  projects,  there  are  several  levels 
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of  management  and  administration;  an  inadequate  information  flow  of  information 
can  adversely  affect  decision  making  about  important  project  tasks.  Care  should 
be  taken  to  insure  that  all  participants  get  the  information  they  need  at  the  time 
they  need  it. 

•  It  is  important  for  the  prime  contractor  to  have  experience  in  administering  a  large 
government  contract. 

•  There  is  a  major  discrepancy  between  an  academic  schedule  and  a  non-academic 
schedule.  The  government  works  on  a  12-month,  5 -day  a  week  calendar  rather 
than  a  9-month  academic  calendar.  Holiday  schedules  and  summer  vacations  are 
not  the  same.  The  difference  in  these  schedules  caused  coordination  problems  that 
were  not  always  easy  to  rectify.  Generally,  community  college  personnel  adjusted 
their  schedules  to  meet  the  contracted  deadlines.  However,  it  is  important  to  keep 
in  mind  that  when  university  or  community  college  faculty  work  with 
organizations  that  operate  on  a  12-month  calendar,  coordination  is  essential  and 
tension  and  friction  can  occur. 

•  No  organization  will  have  complete  expertise  or  knowledge  about  any  project.  In 
fact,  one  reason  several  organizations  are  involved  in  large  projects  is  to  capitalize 
on  their  differing  strengths.  As  contractors  and  subcontractors  try  to  influence 
each  other  in  their  areas  of  expertise,  resistance  can  occur.  This  is  especially  true 
when  one  group  or  the  other  believes  that  their  expertise  is  equal  to  that  of 
another  group.  Understanding  who  has  knowledge  or  lack  of  it  is  important  to 
project  success  and  completion.  Issues  concerning  expertise  can  lead  to  questions 
about  who  should  be  the  prime  contractor  and  who  should  be  the  subcontractor. 

•  The  contractor/subcontractor  relationship  can  be  further  complicated  by  issues  of 
credibility.  Because  the  prime  contractor  is  ultimately  responsible  for  performing 
the  contracted  tasks,  the  prime  can  apply  too  much  control.  If  this  control  is 
perceived  by  the  subcontractor  as  due  to  lack  of  credibility,  it  can  take  a  toll  on 
morale.  Such  a  situation  should  be  avoided  because  it  can  damage  an  otherwise 
good  working  relationship  between  organizations  and/or  individuals. 

Even  though  there  were  issues  that  arose  from  the  contractor/subcontractor  relationship, 
each  group  had  strengths  that  were  needed  to  complete  the  project.  For  example,  experience  in 
distance  education,  ISD,  project  evaluation,  military  course  delivery,  computer  expertise,  technical 
expertise  about  course  production,  delivery,  and  TNET,  and  proficiency  in  teaching  were 
necessary.  The  organization  of  the  roles  and  responsibilities  of  each  group  is  an  important 
management  task  that  had  to  be  planned. 

Project  Evaluation 

The  prime  contractor  was  responsible  for  conducting  the  project  evaluation  and  was  also 
responsible  for  administering  the  contract  and  providing  technical  expertise  to  the  subcontractor 
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in  the  areas  of  distance  education,  instructional  design  and  course  development,  and  military 
instruction.  While  these  multiple  roles  could  have  been  viewed  as  potential  areas  of  conflict,  the 
contractor  had  expertise  in  evaluation.  Performing  multiple  project  roles  is  cost  effective. 
However,  contracting  part  of  the  evaluation  component  to  outside  evaluators  was  used  to 
minimize  potential  conflict  of  interest. 

Community  College  and  Military  Coordination 

The  success  of  the  FTP  was  in  large  measure  a  function  of  the  ability  of  the  community 
college  and  military  to  work  together.  There  were  some  major  challenges  that  existed  for  the 
community  college  faculty  and  personnel  in  their  efforts  to  present  military  instruction  via  VTT: 

•  the  ability  of  the  community  college  faculty  to  deal  with  the  military  culture 

•  issues  surrounding  the  assignment  of  SMEs  and  the  expertise  of  the  SMEs 

•  establishing  a  relationship  between  the  anchor  community  college  and  the 
proponent  schools 

•  obtaining  students  for  the  courses 

•  obtaining  military  course  support  materials  for  the  project. 

Military  Culture.  In  order  to  successfully  complete  the  project  it  was  necessary  for  the 
civilian  employees  of  the  community  college  to  understand  the  military  culture,  rank  structure, 
training  methodology,  and  organizational  hierarchy.  Since  the  project  was  performed  under  a 
contract  with  DoD  it  was  necessary  for  the  community  college  personnel  to  accommodate  the 
military  by  including  leaders  from  die  major  military  commands  (US  2nd  Army,  FLARNG,  81st 
ARCOM,  etc.),  the  proponent  schools,  and  the  local  USARF  schools  in  planning  and  executing 
of  the  project.  In  addition,  the  community  college  personnel  had  to  learn  military  technical  terms 
and  a  myriad  of  acronyms. 

The  community  colleges,  with  assistance  from  DITRA  and  the  prime  contractor,  were  able 
to  successfully  adapt  to  the  military  structure.  DITRA  assisted  the  community  college  by 
working  with  the  various  commands  that  needed  to  coordinate  with  the  college.  The  prime 
contractor  provided  a  military  training  specialist  who  worked  with  the  community  college 
Instructional  Manager  and  the  course  developers  to  provide  training  and  context  for  the  military 
aspects  of  the  course  conversion. 

SME  Issues.  In  order  to  deal  with  the  military  content,  a  SME  was  assigned  to  work 
with  each  community  college  course  developer.  This  conversion  process  required  thorough  and 
precise  presentation  of  content  to  insure  that  the  personnel  who  competed  the  courses  would  be 
able  to  perform  their  duties  under  wartime  conditions  and  to  be  certified  for  the  MOS.  It  was 
essential  for  the  SME  to  have  a  high  degree  of  knowledge  in  the  content  area  and  to  be  able  to 
work  cooperatively  with  the  community  college  course  developer. 


122 


To  ensure  that  SME  had  the  knowledge  and  level  of  cooperation  needed  for  successful 
adaptation  of  the  courseware,  the  duties  and  responsibilities  of  the  SMEs  were  clarified  during 
an  in-progress-review  (IPR)  in  February  of  1992.  Representatives  of  the  entire  chain  of 
command  for  the  SMEs  were  present  at  this  IPR  and  agreed  to  the  duties  and  responsibilities. 

The  SMEs  for  the  Army  MOS  courses  were  qualified  USARF  instructors  provided  by  the 
3391st  USARF  School,  in  Jacksonville,  Florida.  The  SMEs  were  experts  in  the  delivery  of  the 
proponent  school  RC3  materials,  not  in  the  development  of  instructional  materials.  Due  to:  (a) 
the  long  chain  of  command  between  the  Office  of  the  Assistant  Secretary  of  Defense  (OASD), 
TPDC  and  the  3391st  USARF  school,  (b)  funding  issues  (funds  to  pay  and  allowances  for  the 
SMEs  were  provided  by  the  U.S.  2nd  Army)  and  (c)  the  process  for  obtaining  military  orders, 
the  first  SME  was  not  assigned  until  three  months  into  the  time  allotted  for  course 
reconfiguration.  This  caused  considerable  problems  for  the  community  college  course  developers. 

In  addition,  since  the  Army  MOS  course  SMEs  were  reservists  on  drill  status,  it  was 
difficult  to  obtain  them  for  more  than  16  hours  per  month  to  help  with  the  reconfiguration 
process.  This  amount  of  time  proved  inadequate  to  provide  the  level  of  assistance  needed  by  the 
course  developers.  The  16  hour  limitation  occurred  because  the  additional  funds  provided  to  the 
81st  ARCOM  for  assisting  in  the  FTP  were  only  sufficient  to  cover  this  amount  of  time. 

Another  difficulty  with  scheduling  the  SME’s  time  was  that  as  drilling  reservists,  they  had 
to  perform  their  military  duties  in  addition  to  civilian  jobs  and  family  responsibilities.  Two  of 
the  SMEs  (one  Army  and  one  Navy)  lived  in  cities  other  than  Jacksonville,  making  consultation 
and  communication  difficult  and  time  consuming. 

Relationship  between  the  community  college  and  the  proponent  schools.  In  order  to 
provide  a  mechanism  for  official  service  approval  of  the  courses,  it  was  necessary  to  establish 
formal  POCs  with  all  of  the  proponent  schools.  To  accomplish  this,  DITRA  made  initial,  high 
level  contacts  with  the  schools.  From  these  contacts,  introductory  working  level  meetings  were 
held  to  define  procedures  for  approval  of  courseware  and  to  determine  other  project  support  that 
would  be  provided  by  the  proponent  school.  Once  this  initial  contact  between  the  community 
college  Instructional  Manager  and  the  proponent  schools  had  been  established,  another  delay  of 
several  weeks  occurred  due  to  the  time  it  took  to  build  working  relationships  and  to  formalize 
the  course  approval  process.  Since  direct  contact  with  the  proponent  schools  was  necessary  to 
obtain  approval  for  the  Army  MOS  courses,  these  delays  were  problematic. 

Coordination  between  the  anchor  community  college  and  the  proponent  schools  (and 
other  military  agencies,  e.g.,  AETD)  was  accomplished  through  a  formal  Memorandum  of 
Understanding  (MOU)  and  an  informal  memorandum  of  approval  that  accompanied  each 
instructional  unit  sent  to  the  school  for  review.  Final  proponent  school  approval  for  the  complete 
courseware  package  was  sent  to  community  college  Instructional  Manager  by  the  proponent, 
through  DITRA. 

Obtaining  students.  Direct  coordination  with  military  commands  responsible  for 
generating  orders  for  students  to  attend  the  courses  was  a  function  handled  by  DITRA.  The 
orders  process  was  a  source  of  frustration  for  the  community  colleges  throughout  the  course 
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delivery  phase.  Student  orders  were  sometimes  published  only  one  or  two  days  prior  to  the  start 
of  a  course;  some  students  with  orders  never  reported  to  the  community  colleges  and  some 
students  arrived  at  the  community  colleges  with  orders  too  short  to  allow  them  to  attend  the  full 
course.  These  issued  were  resolved  by  the  military  commands  within  the  first  three  days  of  the 
MOS  courses. 

Obtaining  military  course  support  materials.  Obtaining  the  ancillary  course  support 
materials  (Army  manuals,  regulations,  etc)  proved  to  be  a  problem  given  the  time  constraints 
caused  by  the  project’s  delivery  schet  jle.  Army  regulations  specify  a  routine  established 
procedure  for  civilians  to  obtain  materials  in  support  of  government  contracts.  This  standard 
procedure  proved  to  be  inadequate  to  meet  the  short  time  frames  of  the  project.  However,  with 
DITRA’s  assistance,  provisions  were  made  for  the  FTP  to  get  special  priority.  Without  this 
priority,  obtaining  the  materials  and  forms  would  have  been  a  significant  problem. 

In  summary,  the  organizational  and  managerial  issues  that  were  identified  revolved 
primarily  around:  (a)  the  roles  of  the  prime  contractor  and  the  subcontractor  and  (b)  military 
coordination  with  the  community  colleges.  To  summarize  this  objective,  the  civilian  and  military 
organizations  that  participated  in  this  project  worked  together  to  design,  develop,  produce,  and 
deliver  high  quality  instruction  to  military  students.  In  view  of  the  size  of  the  project,  there  were 
few  problems  and  issues  that  were  not  easily  resolved.  Many  of  the  civilians  and  military 
involved  in  the  project  suggested  that  it  was  a  win-win  situation  for  everyone.  Some  of  the  key 
issues  and  lesson  learned  from  the  FTP  were: 

•  Success  depends  on  the  availability  of  military  support  to  the  community  college, 
including  SMEs,  the  satellite  system,  proponent  school  support,  and  sufficient 
students  for  classes. 

•  Professional  instructional  designers  are  needed  to  work  with  community  college 
faculty. 

•  Clear  designations  of  roles  and  responsibilities  for  all  project  personnel  are 
required. 

•  Sufficient  funding  and  lead  time  are  needed  to  design  and  deliver  the  courses. 

•  Ways  to  reconcile  differences  in  scheduling  based  on  an  academic  vs.  government 
calendar  are  necessary. 

•  Informal  as  well  as  formal  mechanisms  for  interaction  among  agencies  and 
institutions  are  needed. 
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Conclusions  about  the  Community  Colleges 


Part  of  the  missions  of  the  community  colleges  in  Florida  is  outreach  to  special  segments 
and  groups  within  the  community.  Military  students  are  part  of  the  community  and  as  such  can 
expect  community  colleges  to  be  providers  of  military  training.  Florida’s  community  colleges 
are  networked  both  formally  and  informally;  they  can  and  do  work  together  regularly. 

Because  community  colleges  strive  to  meet  the  needs  of  their  respective  constituencies, 
they  have  varying  instructional  and  technical  resources.  For  this  project,  only  FCCJ  had  the 
technical  studio  capabilities  to  produce  the  instruction,  but  each  college  had  the  technical 
personnel,  space  requirements,  and  support  equipment  that  were  needed  to  successfully  implement 
the  instruction. 

There  were  different  instructional  requirements  for  the  origination  site  and  for  the  remote 
sites.  Since  the  faculty  at  the  origination  site  were  designing,  developing,  and  implementing  the 
instruction,  it  was  necessary  for  them  to  be  excellent  teachers,  be  knowledgeable  about  distance 
learning,  have  good  ISD  or  course  development  skills,  and  understand  aspects  of  military  training. 
The  origination  site  faculty  were  excellent  teachers,  but  needed  training  in  distance  education, 
reconfiguring  military  courses,  and  ISD.  These  were  provided  and  the  faculty  were  able  to 
deliver  good  courses.  The  remote  site  faculty  needed  context  for  the  project,  but  since  they  did 
not  develop  or  implement  the  courses,  they  did  not  need  training  in  many  aspects  of  the  project 
that  were  required  for  the  origination  site  faculty.  However,  it  can  be  assumed  that  any 
community  college  faculty  would  need  training  in  implementing  a  VTT  approach.  It  can  also 
be  assumed  that  as  professional  educators,  a  comprehensive  staff  development  program  can 
alleviate  any  lack  of  capability  that  community  college  faculty  have  regarding  their  VTT  roles. 

The  staff  training  that  was  provided  by  the  FTP  was  evaluated  as  very  helpful  and 
prepared  the  community  college  with  the  skills  necessary  to  perform  their  roles  and 
responsibilities.  While  extensive  training  was  provided  about  TNET,  the  non-technical 
community  college  faculty  never  felt  fully  prepared  to  operate  the  system  (even  though  they  were 
never  expected  to  operate  TNET  without  the  assistance  of  a  trained  technician). 

The  community  colleges  were  able  to  offer  academic  and  continuing  education  credit  for 
all  five  VTT  courses.  Even  though  the  Army  has  identified  S3  MOSs  that  can  be  trained  by 
civilian  institutions,  none  of  the  MOS  courses  presented  by  the  FTP  were  among  those  listed. 
Still,  the  community  colleges  presented  the  instruction  and  granted  credit.  The  credit  given  was 
not  standard  across  the  three  colleges  because  of  their  individualized  curricula.  While  the  college 
courses  and  the  military  offerings  were  "made  to  fit”  (that  is,  continuing  credit  was  offered),  there 
was  not  a  one-to-one  correspondence  between  community  college  courses  and  military  courses. 
However,  the  students  were  very  favorably  disposed  to  receiving  college  credit  and  it  was  worth 
the  effort  of  the  community  colleges  to  provide  it. 

The  community  colleges  in  this  project  were  organized  through  subcontracts  where  FCCJ 
was  tire  prime  contractor  and  VCC  and  SPJC  were  the  subcontractors.  This  proved  to  be  an 
effective  way  for  the  community  colleges  to  work  together.  Likewise,  formal  contracts  and 
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MOAs  were  established  between  1ST  and  FCCJ,  and  between  FCCJ  and  a  variety  of  military 
agencies,  e.g.,  AETD.  While  there  were  some  problems  and  differences  of  opinion  regarding 
decision  making  and  the  course  development  process,  all  institutions  involved  worked  together 
and  delivered  high  quality  instruction  to  military  students. 

Issues 

It  is  essential  to  have  qualified  and  experienced  technical  and  instructional  personnel 
working  on  a  project  of  this  magnitude.  Only  one  of  the  community  colleges  in  the  FTP  had  the 
technical  capabilities  necessary  to  be  the  origination  site.  If  community  colleges  are  to  be 
providers  of  military  training  using  a  VTT  system,  at  least  one  site  must  have  the  studio 
capabilities  and  have  the  experienced  personnel  to  be  able  to  present  the  instruction. 

Some  observers,  both  civilian  and  military,  staff  and  students,  believed  the  VTT  instructor 
should  not  be  civilian.  They  concluded  there  were  too  many  questions  only  someone  in  the 
military  could  address.  Program  designers  expected  the  military  to  send  complete  POIs  or  syllabi 
to  be  modified  for  delivery,  but  this  was  not  always  the  case.  The  instructor  for  76Y10 
complained  that  in  some  cases  he  was  not  sent  any  lecture  materials;  he  and  the  military  SME 
were  required  to  create  the  course  "from  scratch."  Without  content  expertise,  military  experience, 
and  adequate  course  POIs,  redesigning  and  teaching  military  courses  is  a  difficult  task. 

A  lack  of  military  background  occasionally  led  an  instructor  to  lose  credibility  with  the 
students.  This  experience  gap  was  most  obvious  for  classes  where  there  were  MOS  experienced 
students.  Often  members  of  the  class  knew  more  about  the  course  material  than  did  the 
instructor.  The  76Y10  instructors  even  found  that  the  experienced  students  were  a  valuable 
resource  for  other  students  in  the  course.  Still,  it  should  be  noted  that  military  and  instructional 
coordinators  usually  worked  very  well  together.  In  doing  so,  they  provided  the  most  visible  and 
positive  balance  of  military  and  college  initiatives.  An  issue  though  is  whether  civilians  should 
provide  military  instruction  when  their  content  background  and  military  experience  is  limited. 
The  civilian  educators  did,  however,  bring  other  important  capabilities  to  the  project,  and  each 
project  must  weigh  the  pros  and  cons  of  using  civilians  to  teach  military  students. 

There  was  redundancy  of  skills  at  the  remote  sites  for  the  MOS  courses.  Each  course  had 
a  community  college  representative  and  a  military  representative  and  both  were  needed  so  that 
students  could  receive  credit  and  could  be  MOS  qualified.  This  was  inefficient.  In  future 
endeavors  where  the  community  college  and  military  are  going  to  team  together,  how  the  IC  and 
MSC  roles  can  be  combined  or  interchanged  should  be  examined  to  eliminate  redundancy. 

Perhaps  the  predominate  issue  related  to  staff  training  revolves  around  selecting  versus 
training  personnel.  The  FTP  decided  to  select  community  college  professors  who  had  content 
and  subject  matter  expertise  wherever  possible  rather  than  course  development  (ISD)  expertise. 
However,  since  the  courses  had  considerable  military  content  requirements,  even  the  instructors’ 
content  expertise  was  not  sufficient.  The  lack  of  ISD  skills,  military  content  expertise,  and  TNET 
background  made  the  staff  training  requirements  significant. 
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Another  option  would  have  been  to  separate  the  course  design  responsibilities  from  the 
course  delivery  responsibilities  (e.g.,  hire  course  developers  and  military  SMEs  to  design  and 
develop  the  courses  and  another  group  to  implement  the  courses).  However,  few  community 
college  instructors  have  formal  1SD  skills.  While  this  option  may  have  alleviated  the  need  for 
training  in  course  development,  any  on-camera  teacher,  civilian  or  military,  will  still  need 
technical  and  instructional  training. 

Each  project  should  carefully  consider  the  training  versus  selection  issue.  A  key  decision 
will  be  to  determine  which  skills  are  easiest  or  best  to  provide  training  for  and  which  skills 
should  personnel  possess  when  they  are  selected  to  perform  certain  roles. 

The  instructional  personnel  stated  that  they  needed  more  practice  with  the  technology. 
An  issue  is  how  much  additional  training  would  be  enough.  People  who  fear  technology  may 
never  be  comfortable  with  complex  systems.  Some  questions  to  consider  are  what  kinds  of 
options  for  additional  training  and/or  for  additional  help  are  available,  e.g.,  would  more  OJT  be 
sufficient. 

In  addition,  the  course  developers  stated  that  they  needed  more  practice  and  corrective 
feedback  during  the  course  development  process.  While  better  prepared  materials  from  the 
proponent  schools  would  have  helped,  an  opportunity  for  the  VTT  instructors  to  attend  a 
proponent  school  course  would  have  been  very  helpful.  Although  this  was  considered,  attendance 
was  not  possible  because  the  courses  were  not  offered  during  the  time  frame  of  the  FTP. 

The  military  services  have  made  provisions  for  civilian  institutions  to  provide  military 
occupational  training  on  a  limited  basis.  There  is  no  evidence  to  suggest  that  these  efforts  have 
resulted  in  system-wide  adoption  of  civilian  institutions  as  trainers  of  the  military  services. 
Although  the  military  has  identified  areas  where  civilians  can  provide  military  training,  it  is 
unclear  if  the  military  training  requirements  for  these  areas  could  be  met  with  one  course  or 
whether  an  entire  program  is  needed  to  train  the  service  members  to  the  standards  set  by  the 
proponent  schools. 

Courses  can  be  successfully  taught  by  civilian  institutions  even  though  they  have  not  been 
designated  as  appropriate  for  civilian  contract  training  by  FORSCOM/TRADOC  Regulation 
135-3.  Creating  a  standard  curriculum  for  community  colleges  operating  a  network  designed  for 
delivering  instruction  to  the  military  would  enable  the  colleges  to  standardize  the  type  and 
amount  of  credit  received  for  the  successful  completion  of  military  courses  taken  at  civilian 
institutions. 

College  credit  can  be  issued  for  military  courses  taken  at  a  community  college,  but 
community  colleges  that  are  funded  with  state  tax  dollars  receive  more  full  time  student 
equivalent  (FTE)  dollars  to  deliver  credit  rather  than  non-credit  classes.  There  is  usually  a 
substantial  difference  in  the  amount  of  tuition  paid  by  the  student;  colleges  must  collect  a  fee  that 
includes  all  the  expenses  of  curriculum  development,  design,  and  delivery.  In  short,  community 
colleges  must  recoup  all  their  costs  to  offer  customized  non-college  credit  courses.  Therefore, 


127 


it  would  be  more  cost  effective  for  the  community  colleges  to  deliver  standardized,  college  credit 
courses  than  to  design  new  courses. 


A  customized  associate  degree  program  with  standardized  credits  for  specific  military 
courses  would  simplify  issuing  credit  for  military  courses  taken  through  community  colleges. 
This  could  be  done  by  designating  a  nationwide  community  college  network  for  delivering  VTT 
instruction.  However,  if  uniformity  of  college  credit  is  desired  by  the  military,  the  community 
colleges  will  need  to  negotiate  among  themselves  to  provide  this  service  to  the  military. 

Community  colleges  can  offer  benefits  to  students  that  traditional  military  instructional 
organizations  are  unable  to  provide,  e.g.,  college  credit,  library  facilities,  and  advising.  Plus, 
community  colleges  can  form  teaching-learning  networks  that  may  be  beneficial.  For  example, 
if  community  colleges  could  maintain  standard  and  consistent  training  across  locations,  then  as 
military  students  are  reassigned  to  new  locations  they  could  continue  their  training  without  losing 
time  or  resources.  If  this  is  perceived  as  a  benefit  to  the  DoD,  then  the  ability  of  the  community 
colleges  to  provide  military  instruction  must  be  seen  as  an  advantage. 

Finally,  there  were  multiple  roles  to  be  performed  in  a  project  of  this  size  and  magnitude. 
How  the  roles  are  defined  and  implemented  and  who  wields  ultimate  power  plays  an  important 
part  in  the  success  of  the  project.  The  specific  expertise  of  project  participants  in  relationship 
to  their  assigned  roles  and  responsibilities  needs  to  be  carefully  outlined  and  understood. 

Regarding  the  relationships  between  the  community  colleges  and  military  organizations, 
an  individual  familiar  with  military  training  and  procedures  should  be  readily  available  to  the 
community  college  staff  to  facilitate  civilian-military  cooperation  and  advise  the  community 
college  staff  in  military  matters.  For  example,  more  proponent  school  support  was  needed  for 
community  colleges  in  developing  military  VTT  courseware.  This  support  could  come  in  the 
form  of  SMEs  familiar  with  technical  content  and  having  previous  course  development 
experience.  Contacts  between  the  proponent  schools  and  the  community  colleges  need  to  be 
established  early  in  the  project.  In  addition,  the  SMEs  must  be  readily  available  for  course 
development.  The  FCCJ  Instructional  Manager  estimated  that  reconfiguration  for  a  two  week 
course  required  from  32  to  40  hours  of  SME  time  per  month  during  the  development  phase. 
Sufficient  funding  for  the  necessary  SME  time  must  be  provided  in  future  VTT  reconfiguration 
efforts. 


How  formal  or  informal  these  roles  and  responsibilities  need  to  be  among  agencies  is 
probably  as  dependent  on  the  personalities  of  the  people  involved  in  a  project  as  it  is  a  function 
of  the  contractual  relationships.  It  is  possible  that  internal  civilian  coordination  and 
civilian/military  coordination  could  be  improved  by  the  use  of  a  MOU  between  a  network  of 
community  colleges  and  military  commands.  The  MOU  could  become  the  basic  working 
document  and  should  detail  the  roles  and  responsibilities  of  the  community  college  network  as 
well  as  the  military  organization  (Watt,  1988).  It  is  doubtful,  however,  that  any  formal 
agreement  could  address  all  the  issues  that  will  arise  in  the  course  of  a  large  project  such  as  the 
FTP.  It  is  therefore  of  utmost  importance  to  have  groups  who  are  willing  participants  and  to 
have  people  in  administrative  positions  that  have  as  their  first  priority  the  education  of  the 
military  students. 
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OBJECTIVES  RELATED  TO  COSTS 


The  purpose  of  the  cost  analysis  was  to  address  four  objectives  by  answering  the 
following  research  questions: 

•  What  was  the  cost  of  conducting  the  pilot  test? 

•  What  was  the  cost  of  designing,  developing,  and  implementing  each  course? 

•  What  was  the  total,  non-experimental  cost  of  each  course? 

•  What  was  the  non-experimental  cost  of  each  course  per  student? 

•  What  was  the  non-experimental  cost  of  delivery  per  hour  of  instruction  for  each 

course? 

•  What  was  the  cost  of  each  graphic  used  in  the  each  course? 

•  How  do  the  non-experimental  costs  per  student  compare  to  the  estimated  costs  of 
traditional  military  training? 

•  Was  the  VTT  delivered  by  community  colleges  cost  effective  compared  to 
traditional  military  training? 


Methodology 

Procedure 

The  costs  associated  with  the  project  were  collected  and  categorized.  The  sources  of  cost 
data  were  1ST,  FCCJ,  VCC,  and  SPJC.  Costs  for  the  prime  contractor  and  all  subcontractors 
were  obtained  by  examining  the  actual  billings.  Each  item  from  the  monthly  billings  was 
examined  to  determine  its  relationship  to  the  phases  outlined  in  the  SAT.  The  items  were  also 
categorized  according  to  whether  they  related  to  the  operation  of  a  remote  classroom  site  or  to 
the  administration  of  the  pilot  project  (as  opposed  to  the  operation  of  an  on-going  VTT  network). 
Each  item  was  then  charged  to  a  specific  course  or  courses  and  classified  into:  (a)  one  of  the 
five  conventional  costing  areas:  personnel,  equipment,  supplies,  contracts,  or  travel,  (b)  to  course 
administration. 

The  subcontracts  with  the  participating  community  colleges  called  for  the  delivery  of  the 
instruction  at  a  contractually  negotiated  fixed  price  per  course,  plus  student  registration  and 
tuition  fees.  The  registration  and  tuition  fees  represent  the  per  student  variable  costs  for  each 
course. 
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Since  the  cost  data  were  collected  on  the  basis  of  the  subcontractor  billings  to  1ST,  some 
global  decisions  were  made  in  order  to  deal  with  certain  recurring  charges  in  a  consistent  manner. 
The  results  of  these  decisions  had  some  influence  on  the  final  cost  results  and  this  should  be 
taken  in  to  account  when  making  judgements  based  on  the  cost  data  described  in  this  report. 
Major  decisions  included: 

•  Administration.  Administration  costs  not  directly  related  to  instruction  (e.g., 
reporting  requirements  and  management  functions  related  to  the  contract  with  1ST 
and  administrative  assistance  in  these  matters)  were  divided  equally  among  all 
courses  since  it  was  impossible  to  determine  from  the  subcontractor  billings  the 
particular  course  or  courses  that  were  incurring  the  expense.  The  exception  was 
71L10,  which  incurred  higher  costs  because  of  project  startup  activities.  Upon 
completion  of  the  delivery  of  a  course  and  the  subsequent  revision  of  the 
courseware,  administrative  charges  for  a  particular  course  were  no  longer  made. 
This  resulted  in  administrative  costs  being  charged  to  the  HazWaste  and  TQL 
courses  for  almost  the  entire  length  of  the  project  since  these  courses  were  the  last 
active  courses.  It  also  resulted  in  higher  administrative  charges  to  HazWaste  and 
TQL  during  the  closing  months  of  the  project  since  administrative  costs  remained 
roughly  the  same  throughout  the  project. 

This  procedure  is  consistent  with  the  accepted  practice  of  recognizing 
period  costs  (costs  associated  with  research,  development  and  administration)  as 
expenses  at  the  time  they  are  incurred.  While  this  practice  may  appear  to  be  an 
arbitrary  allocation  of  costs,  otter  methods  of  allocating  costs  that  were  considered 
for  use  (equal  distribution  of  administrative  costs  and  prorating  of  administrative 
costs  based  on  total  course  length)  were  equally  arbitrary  and  more  difficult  to 
implement  uniformly.  The  method  chosen  is  supported  by  Gellein  &  Newman 
(1973). 

•  Local/long  distance  telephone  service  and  facsimile  equipment.  Since  it  was 
contractually  necessary  for  the  project  to  pay  all  of  its  own  costs,  FCCJ  charged 
the  project  for:  (a)  the  installation  and  rental  of  telephone  lines  used  exclusively 
for  the  project,  (b)  local  and  long  distance  telephone  service,  and  (c)  the  monthly 
rental  of  a  fax  machine  dedicated  to  the  project.  These  costs  were  charged  to  all 
courses  as  outlined  in  the  section  on  administrative  costs. 

•  Analysis  of  the  existing  courseware.  There  was  no  job  task  analysis  of  the 
occupations  associated  with  any  of  the  five  courses.  Therefore,  this  phase  of  the 
project  centered  around  an  analysis  of  the  existing  course  materials  and  their 
suitability  for  VTT.  Most  of  this  analysis  was  conducted  in  December  1991  and 
early  January  1992.  The  charges  in  the  Analyze  category  (from  the  SAT  model) 
for  all  courses  reflect  the  salaries  and  travel  costs  of  1ST  and  FCCJ  personnel 
during  this  period. 
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Remote  site  costs.  The  cost  of  leasing  the  TNET  equipment  for  the  remote  sites 
was  charged  to  each  course  on  an  equal  basis;  that  is,  each  course  was  charged  1/5 
of  the  total  cost.  While  this  is  consistent  with  accepted  accounting  practices 
(Gellein  &  Newman,  1973),  it  does  not  reflect  a  cost  that  is  representative  of  the 
number  of  days  each  course  presented  instruction  over  the  system.  Because  lease 
of  the  TNET  system  included  24  hours  a  day  satellite  time,  the  cost  of  system 
rental  for  instructional  days  was  the  same  as  the  cost  for  non-instructional  days. 

The  project  was  not  charged  TNET  rental  for  the  two  remote  sites  at  Camp 
Fogarty  and  Ft  Taylor  Hardin  because  these  sites  had  previously  been  installed  for 
other  military  training.  Therefore,  the  only  remote  site  costs  incurred  by  the 
project  for  these  courses  were  the  community  college  tuition  fees,  shipping  of 
instructional  materials,  and  travel  costs  for  the  1ST  personnel  involved  in  the 
delivery  and  evaluation  of  the  HazWaste  and  TQL  courses  presented  at  these  sites. 
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Objective  D1 

Develop  a  Cost  Model  that  Includes  the  Elements  Necessary  for  Developing  and 
Offering  Teletraining  Courses  from  Community  Colleges 


The  cost  model  used  for  the  cost  analysis  was  adapted  from  Bramble  and  Bauer  (1991). 
The  model  used  the  components  of  the  military  SAT  model  that  is  used  by  the  Army  and  that 
was  adapted  for  use  in  this  project.  The  cost  model  enabled  the  FTP  to  determine  the  total  cost 
of  producing  all  of  the  courses  and  the  cost  of  producing  a  single  course.  The  addition  of  the 
administrative  cost  category  makes  it  possible  to  account  for  the  costs  associated  with  the  overall 
administration  of  a  pilot  test  (one-time  experimental  costs)  as  well  as  the  operation  of  an 
established  VTT  network. 

The  cost  model  used  in  the  FTP  was: 

n 

Ci=g  (ANi  +DEi  +DVi  +DLi  +EVi  +Sum { SOji)  +ADi) 


The  variables  in  this  model  are  defined  as  follows: 

•  Ci  =  the  total  cost  of  course  "i" 

•  ANi  =  the  cost  of  analyzing  course  ”i" 

•  DEi  =  the  cost  of  designing  course  "i" 

•  DVi  =  the  cost  of  developing  course  "i" 

•  DLi  =  the  cost  of  delivering  course  "i" 

•  EVi  =  the  cost  of  evaluating  course  "i" 

•  SOji  =  the  cost  of  operating  remote  site  "j”  for  course  "i" 

•  ADi  =  administrative  costs  associated  with  course  "i”. 
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Objective  D2 

Determine  the  Cost  of  Each  Course  During  the  Pilot  Test 


The  purpose  of  this  objective  was  to  determine  the  cost  of  conducting  the  pilot  test 
including  the  costs  of  project  evaluation  and  administration.  Using  the  cost  model,  Table  35  was 
developed  to  present  die  total  overall  costs  of  the  project.  The  costs  are  categorized  by  the  five 
phases  of  the  SAT  model,  the  administration  costs,  the  operating  costs  at  each  remote  site,  and 
the  aggregate  cost  for  all  the  remote  sites. 


Table  35 

Cost  of  the  Pilot  Test 


ANALYZE 

$1,315.07 

$1,717.17 

$1,315.07 

$879.20 

$471.66 

DESIGN 

$14,377.00 

$12,326.97 

$13,024.20 

$10,273.06 

$11,800.72 

DEVELOPMENT 

$155,846.43 

$102,556.57 

$113,916.09 

$52,644.25 

$51,076.78 

TOTAL  COST  OF 
DEVELOPMENT 

$171,538.50 

$125,600.71 

$128,255.36 

$63,796.51 

$63,399.16 

DELIVERY 

$29,551.70 

$28,831.29 

$29,157.47 

$31,882.55 

$31,290.91 

EVALUATION 

$32,516.76 

$28,873.30 

$27,896.97 

$26,787.06 

$29,087.83 

ADMINISTRATION 

$39,353.80 

$37,870.75 

$40,027.68 

$43,187.52 

$39,681.18 

SITE  COSTS: 

SPJC 

$16,561.62 

$15,443.39 

$24,971.43 

$24,230.35 

$23,674.75 

VCC 

$16,279.39 

$15,222.56 

$24,840.52 

$23,403.27 

$23,298.03 

FCCJ2 

$14,183.96 

$10,356.02 

$18,160.96 

$9,474.82 

$9,404.02 

FTH 

$392.00 

FOG 

$428.00 

$316.00 

TOTAL  SITE  COST 

$47,024.97 

$41,021.97 

$67,972.91 

$57,928.44 

$56,692.80 

TOTAL 

$319,985.73 

$253,198.02 

$293,310.39 

$223,582.08 

$220,101.88 

As  shown  in  the  table,  the  cost  of  conducting  the  pilot  test  for  each  course,  whether  it  was 
a  one-day  or  10-day  course,  fall  within  a  $100,000  range.  This  difference  is  due  primarily  to  the 
1ST  and  FCCJ  administrative  costs  being  uniformly  distributed  across  the  five  courses,  that  is, 
the  costs  for  conducting  the  pilot  test  were  not  proportional.  Therefore,  the  cost  of  conducting 
a  pilot  test  for  a  10-day  course  is  not  ten  times  greater  than  the  cost  of  conducting  a  pilot  test 
for  a  one-day  course. 
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Objective  D3 

Estimate  the  Costs  of  Course  Design  and  Delivery  that  Would  Result  from  an 
Operational,  Rather  than  Experimental,  VTT  System 


The  purpose  of  this  objective  was  to  separate  the  cost  of  conducting  the  pilot  test  from 
the  costs  of  developing  and  delivering  the  five  FTP  courses  under  the  specific  parameters  of  the 
project,  e.g.,  the  number  of  sites  and  students.  Presented  in  this  objective  are:  (a)  the 
experimental  and  non-experimental  costs  for  conducting  each  course,  (b)  the  cost  of  VTT  delivery 
per  student,  (c)  the  cost  of  delivery  per  instructional  hour,  (d)  the  cost  per  student  hour,  (e)  the 
cost  for  design  and  development  per  hour  of  each  course,  (f)  the  costs  of  different  tasks  related 
to  development  of  each  course,  e.g.,  word  processing  and  graphics,  and  (g)  a  projection  of  the 
cost  of  producing  a  course  given  certain  parameters. 

Experimental  vs.  Non-Experimental  Costs 

One-time  experimental  costs  were  defined  as  costs  associated  with  the  maintenance  and 
dual  management  of  the  instructional  design  and  development  functions  at  both  1ST  and  FCCJ. 
Typical  examples  of  these  costs  were  the  salaries  of  project  managers  at  both  1ST  and  FCCJ  and 
of  an  instructional  designer  at  1ST  to  review  and  approve  the  course  development  that  was 
subcontracted  to  FCCJ.  In  an  on-going  VTT  system,  external  project  administration  might  not 
exist  or  might  not  be  as  extensive.  These  one-time  experimental  costs  were  subtracted  from  the 
total  cost  of  the  pilot  test.  Table  36  shows  the  experimental  costs  associated  with  each  phase  of 
the  project 


Table  36 

Experimental  Costs 


1 

v. -r-  •••>  v-  v  '  .  '  .  ;  '  •  "  'v  r': 

H  ANALYZE 

$1,315.07 

$1,315.07 

$1315.07 

$879.20 

$471.66 

DESIGN 

$2,797.67 

$2,979.77 

$3,669.90 

$482.96 

$789.14 

DEVELOPMENT 

$47,269.95 

$23,322.46 

$24,229.26 

$4381.30 

$4,457.41 

TOTAL  COST  OF 
DEVELOPMENT 

$51,382.69 

$27,617.30 

$29,214.23 

$5,943.46 

$5,718.21 

DELIVERY 

$1,290.65 

$1,070.46 

$2,057.51 

$2,439.21 

$3,996.64  | 

EVALUATION 

$31,077.46 

$26,777.09 

$25,803.78 

$24,731.56 

$27,099.69 

ADMINISTRATION 

$17,368.00 

$17,494.62 

$17,208.48 

$17,255.36 

TOTAL 

$101,118.80 

$72,959.47 

$74,284.00 

$50,369.59 

$51,405.55  | 
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As  previously  stated,  the  71L10  experimental  costs  are  higher  than  the  costs  for  the  other 
two  MOS  courses  of  similar  length  because  71L10  was  the  first  course  developed.  The  71L10 
course  was  used  as  the  prototype  for  designing  the  processes  and  products  (e.g.,  forms)  that  were 
used  during  course  design  and  approval.  While  these  guidelines  were  later  used  for  all  the 
courses,  the  initial  costs  were  charged  to  71L10  because  the  guidelines  were  developed  during 
this  course. 

The  experimental  costs  were  subtracted  from  the  total  costs  of  the  courses  to  calculate  the 
non-experimental  costs  of  each  course.  These  costs  are  shown  in  Table  37.  Delivery  refers  to 
the  origination  site  costs  (FCCJ1)  and  the  cost  of  producing  the  student  and  instructional 
personnel  manuals.  These  costs  were  distributed  equally  across  the  courses  as  discussed  in  the 
Methodology  section.  The  HazWaste  and  TQL  courses,  while  less  expensive  to  develop  because 
they  were  one-day  courses,  were  not  proportionally  less  expensive  overall  because  they  were 
presented  multiple  times,  thus  increasing  the  student  tuition  costs  at  the  remote  sites. 


Table  37 

Noa-Experimeatal  Course  Costs 


ANALYZE 

$0.00 

$402.10 

$0.00 

$0.00 

$0.00 

DESIGN 

$11,579.33 

$9347.20 

$9354.30 

$9,790.10 

$11,011.58 

DEVELOPMENT 

$108,576.48 

$79334.11 

$89,686.83 

$48,062.95 

$46,619.37 

TOTAL  COST  OF 
DEVELOPMENT 

$120,155.81 

$88,983.41 

$99,041.13 

$57,853.05 

$57,630.95 

DELIVERY 

$28,261.05 

$27,760.83 

$27,099.% 

$29,443.34 

$27394.27 

EVALUATION 

$1,439.30 

$2,096.21 

$2,093.19 

$2,055.50 

$2,078.19 

ADMINISTRATION 

$21,985.80 

$20,376.13 

$22,819.20 

$25,932.16 

$25,090.12 

SITE  COST: 

SPJC 

$16,561.62 

$15,443.39 

$24,971.43 

$2433035 

$23,674.75 

VCC 

$16,279.39 

$1532236 

$24,840.52 

$23,403.27 

$23,298.03 

FCCJ2 

$14,183.96 

$10,356.02 

$18,160% 

$9,474.82 

$9,404.02 

MONTGOMERY 

$392.00 

FOG 

$428.00 

$316.00 

TOTAL  SITE  COST 

$47,024.97 

$41,021.97 

$67,972.91 

$57,928.44 

$56,692.80 

TOTAL  COST 

$218,866.93 

$180,238.55 

$219,02639 

$173,212.49 

$168,786.33 
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Analysis  of  the  Non-Experimental  Costs 


In  order  to  compare  the  costs  of  VTT  courses  presented  at  the  community  colleges  with 
traditional  methods  of  military  instruction  (these  comparisons  will  be  presented  in  Objective  D4), 
a  series  of  analyses  were  performed.  These  analyses  break  the  non-experimental  cost  down 
according  to  the  number  of  students,  the  cost  per  course  delivery  hour,  the  delivery  cost  per 
student  hour,  and  the  costs  of  different  aspects  of  course  development. 


Cost  of  VTT  Delivery  per  Student.  In  order  to  determine  the  per  student  cost  of 
delivering  the  training,  the  total  non-experimental  delivery  costs  (see  Table  37)  were  divided  by 
the  total  number  of  students  in  each  course.  The  non-experimental  cost  of  delivery  per  student 
is  shown  in  Table  38. 


Table  38 

Per  Student  Noo- Experimental  Cost  of  Delivery 


Because  site  operating  costs  remained  stable  relative  to  the  student  population  at  the  site, 
the  per  student  cost  of  delivery  was  influenced  by  the  total  number  of  students  who  took  the 
course.  While  the  delivery  of  VTT  is  more  cost  effective  for  large  groups,  having  large  classes 
may  not  be  more  instructionally  effective.  That  is,  the  optimum  class  size  is  not  dependent  on 
economics  alone.  Educational  considerations  such  as  instructor  to  student  ratio  and  the  level  of 
interactivity  must  be  taken  into  account. 

Cost  of  Delivery  per  Hour  of  Instruction.  In  order  to  determine  the  cost  of  the  course 
per  hour  of  instruction,  the  total  non-experimental  delivery  cost  was  divided  by  the  total  number 
of  instructional  hours.  These  costs  are  presented  in  Table  39.  The  average  cost  of  delivery  per 
hour  of  instruction  for  all  five  VTT  courses  was  $660.49. 
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Table  39 

Noo-Experimeatal  Coat  of  Delivery  per  Instructional  Hour 


71L10 

73 

$387.14 

76Y10 

101 

$274.86 

9SB10 

66 

$410.60 

HAZ 

8 

$3,680.42 

TQL 

6 

$4,549.05  g 

As  before,  the  site  operating  costs  remained  stable  in  relation  to  the  amount  of  instruction 
delivered.  Hence,  the  more  hours  of  instruction  delivered,  the  lower  the  cost  per  hour.  This 
explains  the  relatively  high  cost  of  the  two  one-day  special  topics  courses. 


Delivery  Cost  per  Student  Hour.  In  order  to  determine  if  VTT  will  achieve  an  economy 
of  scale  (Dolan,  1977)  relative  to  delivery  cost  per  hour  of  instruction  (i.e.,  will  long  range 
average  costs  decrease  as  the  number  of  students  increases),  it  was  necessary  to  calculate  the 
average  delivery  cost  per  student  per  hour  of  instruction.  This  amount  was  calculated  by  dividing 
the  total  non-experimental  delivery  cost  by  the  total  instructional  hours  multiplied  by  the  total 
number  of  students  (total  non-experimental  delivery  cost/(total  number  of  instructional  hours  x 
total  number  of  students)).  Table  40  shows  the  average  delivery  cost  per  student  hour  for  each 
course. 


Table  40 

Cost  of  Delivery  per  Student  Hour 


TQL  was  the  most  expensive  course  per  student  hour  because  the  course  was  short  (six 
instructional  hours)  and  had  only  60  students.  Although  there  were  more  students  in  TQL  than 
in  the  MOS  courses,  the  length  of  the  MOS  courses  made  them  less  expensive  per  student  hour. 


Analyze,  Design,  and  Development  Costs  per  Instructional  Hour.  In  order  to  provide 
a  comparison  between  the  reconfiguration  of  VTT  courseware  by  the  community  college  and  the 
costs  of  traditionally  developed  courseware,  the  cost  of  analyzing,  designing  and  developing  each 
course  per  hour  of  instruction  was  calculated.  For  this  analysis,  the  total  non-experimental 
analysis,  design,  and  development  costs  were  divided  by  the  total  instructional  hours.  These  costs 
are  presented  in  Table  41. 


137 


Table  41 

Analysis,  Design  and  Development  Casts  per  Hour 


71L10 

73 

$1,645.97 

76Y10 

101 

$881.02 

95B10 

66 

$1,500.02 

HAZ 

8 

$7,231.63 

I  tql 

6 

$9,605.16 

The  high  costs  of  Haz Waste  and  TQL  were  due  to  the  fact  that  these  were  eight  and  six 
hour  courses  respectively.  Therefore,  the  short  courses  were  more  expensive  to  develop  per 
instructional  hour.  The  relatively  low  cost  of  76Y10  for  analysis,  design,  and  development  per 
course  hour  is  due  to  the  fact  that  it  had  more  hours  than  the  other  four  courses. 


Cost  Analysis  of  Personnel  Hours  for  Development  of  Courses.  The  development 
process  can  be  broken  down  into  tasks  or  functions  that  have  to  be  completed  for  the  courses  to 
be  delivered.  For  example,  there  were  salaries  for  word  processors,  graphics  designers, 
instructional  designers,  and  the  instructional  manager.  The  number  of  hours  required  for  each 
function  and  the  salaries  per  hour  were  provided  by  the  FCCJ  Instructional  Manager. 

There  were  two  phases  of  the  course  development  process:  initial  development  and 
revision  of  the  courses  after  delivery.  Both  initial  development  and  revision  costs  for  the 
development  tasks  are  presented  in  Tables  42  and  43.  These  costs  do  not  include  the  cost  of 
reproducing  the  final  copies  of  the  materials  for  distribution,  nor  the  cost  of  shipping  the 
materials  to  the  remote  sites.  These  costs  were  charged  to  the  delivery  phase  instead  of  the 
development  phase. 


Table  42 

Personnel  Costs  Daring  Initial  Coarse  Development 


71LI0 

$51,763.12 

$12,904.73 

$7,689.97 

$15,353.90 

$24,187.36 

76Y10 

$37,631.08 

$9,387.75 

$5479.26 

$11,15842 

$18,277.19 

95B10 

$43,875.22 

$10,963.85 

$6,507.00 

$13,023.91 

$20,640.17  | 

|  HAZ 

$22,267.64 

$5419.18 

$3497.62 

$6,595.25 

$15,730.24  | 

1  TQL 

$21,749.92 

$5,434.60 

$3,186.99 

$6,448.90 

$17,398.78  | 
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Table  43 

Personnel  Costs  for  Revision  After  VTT  Course  Delivery 


HH 

71L10 

$3,160.10 

$2,439.16 

$1,297.68 

$961.25 

$408.53 

76Y10 

$2,642.81 

$2,037.72 

$1,094.50 

$800.85 

$355.93 

95B10 

$1425.23 

$1,168.69 

$625.60 

$465.49 

$237.70 

HAZ 

$1421.54 

$1,139.71 

$636.38 

$509.11 

$636.38 

TQL 

$1,164.15 

$933.62 

$461.04 

$386.13 

$466.81 

As  shown  in  Table  42,  the  word  processing  costs  for  initial  development  of  the 
courseware  were  considerably  higher  than  any  other  costs  associated  with  development,  e.g.,  the 
salaries  of  instructional  designers,  course  developers,  and  the  instructional  manager.  While  the 
costs  of  word  processing  were  also  the  highest  costs  during  revision,  they  were  not 
proportionately  as  high  as  they  were  during  initial  development.  One  reason  for  this  is  that  the 
project  was  only  tasked  to  make  corrections  to  the  courses  and  not  to  conduct  a  full-scale 
revision. 

Cost  of  Graphics.  VTT  instruction  is  highly  dependent  on  graphics.  The  use  of  graphics 
is  desirable  when  using  TNET  because  of  its  two-way  video  capability.  Therefore,  graphics  were 
used  extensively  in  all  the  courses  as  shown  in  Table  44. 

Prior  to  developing  the  graphics,  FCCJ  negotiated  a  fixed-fee  contract  price  of  $10.00  per 
graphic  for  all  of  the  graphics  produced  for  the  project.  However,  by  using  the  costs  from  Tables 
42  and  43,  it  is  possible  to  estimate  the  actual  average  cost  of  developing  and  revising  each 
graphic  used  in  a  course.  In  order  to  calculate  these  costs,  the  total  graphics  development  costs 
for  a  course  were  divided  by  the  total  number  of  graphics  used  in  each  course.  The  cost  per 
graphic  for  each  course  is  shown  in  Table  44. 


Table  44 

Cost  of  Individual  Graphics 


Course 

71L10 

1200 

$10.75 

$2.03 

76Y10 

1180 

$7.96 

$1.73 

95B10 

880 

$12.46 

$1.33 

HAZ 

211 

$26.16 

$5.40 

TQL 

220 

$24.70 

$4.24 

The  actual  costs  for  initially  developing  the  graphics  for  the  MOS  courses  averaged  about 
$10.00  each,  which  was  the  fixed  fee  contract  price  that  was  negotiated.  However,  the  cost  of 
producing  each  graphic  for  the  special  topics  courses  was  considerably  higher  because  there  were 
!  fewer  graphics. 

i 

Cost  Projections 

With  the  data  from  the  previous  analyses,  it  was  possible  to  project  the  cost  of  designing, 
developing,  and  delivering  a  course  under  non-experimental  conditions  given  certain  parameters. 
Typical  examples  of  these  parameters  are:  the  number  of  students,  the  length  of  the  course,  the 
number  of  sites,  the  number  of  graphics,  etc. 

Two  analyses  were  conducted.  The  first  analysis  is  for  a  course  similar  to  the  MOS 
courses  presented  by  the  FTP  (e.g.,  one  that  is  approximately  ten  days  in  length).  The  projection 
provides  a  range  of  costs  from  the  lowest  to  the  highest  incurred  by  the  FTP.  The  projection  also 
provides  the  average  of  the  total  costs.  The  costs  used  in  this  analysis  include  the  cost  of 
designing,  producing,  and  delivering  a  course  to  three  remote  sites.  This  analysis  does  not 
include  any  revision  costs.  The  second  analysis  was  calculated  for  a  course  that  was 
approximately  five  days  in  length  and  delivered  to  four  remote  sites,  as  were  the  special  topics 
courses. 

The  two  generic  courses  are  described  below.  The  cost  projections  come  from  the  tables 
|  in  the  previous  analysis.  The  source  table  for  the  costs  are  listed  in  parentheses  next  to  each  cost 

item. 


MOS-type  Course.  First,  given  a  course  that  has  45  students,  three  TNET  remote  sites 
and  one  origination  site,  80-hours  of  instruction,  and  1087  graphics,  the  expected  costs  would  be 
as  shown  in  Table  45.  Table  45  gives  both  the  high  and  the  low  costs  for  each  area. 


Table  45 
Cost  Projection  1 


|  Analyze,  design,  and  develop  (exc 

2ffit 

wm  H  fill ft&i  /  * 

eluding  graphics)  (Table  37) 

$63,046.07  -  $1 17,535.43 

Delivery  (Table  40) 

$24,732.00-  $58,572.00 

Remote  Site  Costs  (Table  37) 

$33,287.21  -  $112,371.44 

Graphics  (Table  44) 

$8,652.52  -  $13,544.02 

Total  Costs 

$129,727.80  -  $302,022.89 

|  Average  Cost 

$215,870.95 

These  figures  assume  that  only  one  ten-day  course  is  presented  per  month  as  was  done 
during  the  FTP.  This  is  an  underutilization  of  TNET  because  a  maximum  of  three  10-day 
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courses  could  be  presented  in  a  month.  Using  TNET  for  multiple  courses  in  a  month  means  that 
each  course  would  only  pay  for  the  cost  of  use  on  the  days  that  it  was  presented.  Therefore,  the 
total  TNET  cost  would  be  shared  by  all  courses.  That  would  mean  a  reduction  of  approximately 
14%  in  the  delivery  and  remote  site  costs  for  each  course. 

However,  if  the  same  course  was  presented  three  times  per  month,  not  only  would  the 
TNET  costs  be  reduced  as  described  above,  but  the  analyze,  design,  and  development  costs  would 
be  incurred  for  the  first  iteration  of  the  course  only.  This  would  result  in  a  savings  of 
approximately  $72,000.00  per  course  for  the  second  and  third  delivery  of  the  course.  This 
savings,  combined  with  the  savings  resulting  from  a  fully  utilized  TNET  system,  would  result 
in  an  overall  cost  reduction  of  47%  in  subsequent  courses. 

On  the  other  hand,  if  it  is  intended  that  a  course  would  be  presented  only  once  or  twice, 
the  development  would  be  significantly  scaled  down,  thus  reducing  costs.  It  is  impossible  to 
determine  precisely  the  cost  reduction  because  the  reduction  would  be  dependent  on  the  type  of 
course  offered.  However,  it  could  be  expected  that  fewer  media  (e.g.,  graphics,  videotapes) 
would  be  developed,  on-line  and  off-line  activities  would  be  simplified,  and  less  time  would  be 
devoted  to  organizing  administrative  and  managerial  functions.  All  of  these  modifications  would 
decrease  the  VTT  costs. 


Special  topics-type  course.  Second,  given  a  course  that  has  60  students,  four  TNET 
remote  sites  and  one  origination  site,  40-hours  of  instruction,  and  500  graphics,  the  costs 
presented  in  Table  46  could  be  expected: 

Table  46 
Cost  Projection  2 


. ~4;> .v. ~ . .*.*** 
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Analyze,  design,  and  develop  (excluding  graphics) 

$31,523.00-  $58,767.60 

Delivery 

$16,488.00-  $39,048.00 

Remote  Site  Costs 

$44,383.00  -  $149,828.40 

Graphics 

$3,980.00  -  $6,230.00 

Total  Costs 

$96,374.00  -  $253,874.00 

Average  Cost 

$175,124.00 

Again,  this  result  assumes  underutilization  of  TNET.  Presumably,  full  use  of  TNET 
would  allow  delivery  of  six,  five-day  courses.  If  the  same  course  was  offered  six  times,  the  costs 
would  be  much  lower. 

In  summary,  the  purpose  of  Objective  D3  was  to  differentiate  between  the  experimental 
and  non-experimental  costs  of  the  FTP.  While  the  experimental  and  the  non-experimental  costs 
were  separated,  the  non-experimental  VTT  costs  are  inflated  because  of  the  expense  of  leasing 
the  TNET  system  and  due  to  the  high  costs  of  the  community  college  remote  site. 


141 


Objective  D4 

Compare  the  Estimated  Operational  Course  Design  and  Delivery  Costs  to  the  Cost  of 

Selected  Conventional,  Training  Options 

The  purpose  of  this  objective  was  to  determine  whether  or  not  VTT  instruction  was  more 
cost  effective  than  traditional  options.  Given  this  data,  decisions  can  be  made  about  the  use  of 
VTT  in  the  context  of  other  training  options.  While  the  project  was  unable  to  obtain  specific 
data  related  to  the  costs  associated  with  developing  resident  school  courses,  some  assumptions 
could  be  made  based  on  information  from  current  general  government  contract  airfares  and 
Continental  United  States  (CONUS)  per  diem  rates.  Travel  costs  were  obtained  from  the  current 
military  pay  scales  and  the  JTR.  The  FLARNG  indicated  that  to  conserve  resources  some  reserve 
commands  now  send  students  to  two-week  courses  in  an  Annual  Training  (AT)  status.  This 
allows  the  commands  to  pay  a  maximum  of  two-days  per  diem  rather  than  for  the  length  of  the 
course.  It  also  eliminates  the  need  to  pay  the  basic  allowance  for  quarters  (BAQ)  or  the  basic 
allowance  for  subsistence  (BAS). 

The  following  assumptions  were  made  for  all  the  courses  in  order  to  conduct  these 
comparison  analyses: 

•  Air  fare  was  based  on  Orlando,  Florida  to  St.  Louis,  Missouri  round  trip  at  the 
current  government  rate  ($576.00) 

•  Standard  CONUS  per  diem  rate  for  resident  school  travel  is  $66 

•  Local  per  diem  rate  for  VTT  is  $26 

•  Military  instructor  costs  were  based  on  the  1994  pay  scale,  E7  (over  1 1  years  of 
service)  pay  rate,  full  BAQ  (with  dependents)  and  BAS 

•  For  resident  training,  there  were  no  delivery  costs  other  than  instructor  salary  and 
no  site  costs  other  than  base  operating  costs;  were  not  included  in  this  analysis 

•  All  VTT  students  lived  within  a  50-mile  radius,  did  not  require  temporary 
billeting,  and  received  the  local  per  diem  rate. 

The  costs  of  presenting  VTT  at  the  community  college  used  in  this  comparison  were  taken 
from  the  costs  shown  in  Table  37.  The  cost  comparison  of  each  of  the  five  courses  is  shown  in 
Tables  47  and  48.  Any  additional  assumptions  that  were  made  for  each  course  are  included  and 
explained. 

MOS  Courses 

The  comparison  of  the  resident  and  VTT  courses  for  71L10,  76Y10,  and  95B10  are 
included  below.  Additional  assumptions  regarding  the  MOS  courses  are  that:  (a)  there  were 
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three  remote  and  one  origination  TNET  sites,  (b)  the  students  attended  the  resident  course  in  AT 
status,  and  (c)  travel  to  the  resident  school  is  one  day  each  way  with  no  overnight  stay. 


Table  47 

MOS  Courses  Cost  Comparison 
Resident  verms  Teletraining 


ICSIil 

WfS&S 
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71L10 

Resident 

4,356.00 

18,810.00 

1,238.30 

24,404.30 

VTT 

28,760.83 

47,024.97 

5,460.00 

81,245.80 

76Y10 

Resident 

5,280.00 

23,040.00 

1,238.30 

29,558.30 

VTT 

27,760.83 

41,021.97 

1,040.00 

69,822.80 

95B10 

Resident 

3,432.00 

14,976.00 

1,238.30 

19,646.30 

VTT 

27,099.96 

67,972.91 

9,464.00 

104,536.87 

The  total  cost  of  each  option  was  divided  by  the  number  of  students  that  were  enrolled 
in  each  course  (71L10,  N=33;  76Y10,  N=40;  95B10,  N=26).  In  the  71L10  course,  the  estimated 
resident  costs  were  $739.52  per  student  and  for  the  VTT  version  of  the  course  they  were 
$2,461.99  per  student.  In  the  76Y10  course,  the  estimated  resident  costs  were  $738.96  per 
student  and  for  the  VTT  course  they  were  $1,745.57  per  student.  In  the  95B10  course,  the 
estimated  resident  costs  were  $755.63  per  student  and  for  the  VTT  course  they  were  $4,020.65 
per  student. 

Special  Topics  Courses 

The  comparison  of  the  resident  and  VTT  courses  for  HazWaste  and  TQL  are  included  in 
Table  48.  Additional  assumptions  regarding  these  courses  are  that:  (a)  there  were  two  out-of- 
state  remote  sites,  (b)  travel  to  the  resident  school  is  one  day  each  way  with  two  overnight  stays, 
and  (c)  lodging  costs  are  $50  per  student  per  night. 
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Tabic  48 

Special  Topics  Courses  Cost  Comparison 
Resident  versus  Teletraining 


v-  '  ' W  V 

y0:J  1’  .  .  ' 

i  . 

HAZ 

• 

Resident 

22,968.00 

66,816.00 

11,600.00 

88.45 

101,472.45 

VTT 

29,443.34 

57,928.44 

3,016.00 

90.387.78 

|  TQL 

|  Resident 

11,880.00 

34,560.00 

6,000.00 

88.45 

52,528.45 

|  VTT 

27,294.27 

57,928.44 

1,560.00 

86,782.71 

The  total  cost  of  each  option  was  divided  by  the  number  of  students  that  were  enrolled 
in  the  courses  (HazWaste,  N=116;  TQL,  N=60).  The  resident  costs  of  Haz Waste  were  $874.76 
per  student  and  for  the  VTT  course  they  were  $779.21  per  student.  The  resident  costs  of  TQL 
were  $875.47  per  student  and  for  the  VTT  course  they  were  $1,446.38  per  student. 

Analysis  of  Comparison 

Only  the  HazWaste  course  provided  a  cost  savings  over  traditional  methods  of  military 
training.  This  finding  supports  the  Navy’s  guidelines  for  selecting  courses  for  VTT:  courses 
should  be  shorter  rather  than  longer,  and  have  high  throughput.  Significant  costs  for  the  VTT 
delivery  were  those  relating  to  the  use  of  the  community  colleges  to  deliver  the  instruction. 
Because  the  community  colleges  had  to  cover  all  the  costs  they  incurred,  a  fixed-price  was 
negotiated  by  FCCJ  for  the  cost  of  rooms,  equipment,  etc.  These  costs  made  the  VTT  option 
more  expensive  because  such  costs  at  the  resident  school  were  not  included  in  the  analysis. 

As  stated  in  the  assumptions  at  the  beginning  of  this  objective,  base  operating  costs  were 
not  calculated  into  the  resident  school  training  option.  Only  instructor  salary  and  student  travel 
costs  were  included.  This  obviously  skews  the  resident  training  costs  making  them  appear 
considerably  less  expensive  than  they  actually  are.  This  fact  makes  the  cost  comparison  a  rough 
estimate  of  two  very  diverse  training  options  where  all  the  cost  components  could  not  be  directly 
assessed  and  therefore  could  not  be  compared. 

Additionally,  the  TNET  site  cost  is  $7600  per  month  per  site.  This  rate  increased  the  cost 
of  VTT  delivery  tremendously.  If  VTT  were  presented  at  a  military  facility,  the  TNET  cost  must 
still  be  paid,  making  a  VTT  resident  course  more  expensive  than  the  costs  reported  above. 

In  conclusion,  the  costs  paid  by  the  project  for  tuition,  TNET  equipment,  and  use  of  the 
community  colleges’  facilities  greatly  increased  the  price  of  this  training  option  over  traditional 
resident  training.  Whether  or  not  the  community  college  atmosphere  and  granting  of  college 
credit  are  worth  the  additional  costs  are  issues  that  should  be  taken  into  consideration. 
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Conclusions  About  Cost 


Four  cost  objectives  were  investigated.  A  cost  model  was  adapted  based  on  the  model 
described  in  Bramble  and  Bauer  (1991).  The  phases  of  the  SAT  model  (e.g.,  design  and 
development),  plus  site  costs,  administrative  costs,  and  evaluation  costs  were  included  in  the  cost 
model. 


Using  the  cost  model,  the  experimental  and  non-experimental  costs  of  the  project  were 
determined.  The  costs  of  conducting  the  pilot  test  (e.g.,  the  experimental  costs)  for  the  five 
courses  ranged  from  $220,101.88  to  $319,985.73.  The  non-experimental  costs  for  designing, 
delivering,  and  implementing  the  five  courses  ranged  from  $168,786.37  to  $219,036.39. 

When  VTT  instruction  (non-experimental  cost)  was  compared  to  conventional  training 
options,  it  was  cost-effective  in  only  one  instance.  However,  as  previously  stated,  the  resident 
training  costs  were  underestimated  and  the  VTT  costs  were  inflated.  When  the  VTT  HazWaste 
course  was  compared  to  HazWaste  resident  training,  the  VTT  course  cost  approximately  $100 
less  per  student.  This  decrease  was  due  primarily  to  the  fact  that  there  were  more  students  in 
the  VTT  course  and  because  the  resident  course  was  assumed  to  require  that  students  live  in  non¬ 
government  quarters. 

However,  for  the  other  four  courses,  the  VTT  course  was  more  expensive.  In  the  most 
extreme  case,  the  95B10  course  cost  approximately  $3250  more  per  student  than  the  resident 
course.  This  increase  was  due  to  the  fact  that  there  were  few  students  in  this  course  and  that  the 
majority  of  the  students  were  out-of-state.  Out-of-state  tuition  costs  for  these  students  increased 
the  costs  for  95B10.  However,  by  either  presenting  three  different  courses  in  one  month  or  the 
same  course  three  times  in  one  month,  course  design  and  development  costs  can  be  reduced  and 
the  TNET  system  can  be  fully  used.  This  can  result  in  a  cost  savings  of  between  14%  and  47%. 

As  discussed  in  both  the  cost  and  technology  sections  of  this  report,  courses  that  are  short 
and  have  high  demand  or  throughput  are  more  cost  effective  than  others.  The  cost  comparison, 
specifically  the  cost  of  the  HazWaste  course,  supports  this  conclusion. 

Issues 

Even  though  the  experimental  costs  for  the  project  were  removed  when  the  cost  analysis 
was  performed,  the  fact  that  an  experiment  was  being  conducted  influenced  the  costs  obtained 
for  VTT.  First  and  foremost,  the  cost  of  leasing  the  TNET  system  are  significant:  $7600  per 
month  per  site.  While  in  some  circumstances  it  might  be  cost  effective  to  purchase  a  VTT 
system,  the  technology  changes  rapidly.  Purchase  of  a  system  that  becomes  outdated  and  then 
must  be  replaced  or  updated  may  cost  more  in  the  long  run  than  rental  costs.  However,  in 
general,  the  technology  costs  should  decrease  in  the  future. 

The  rental  of  TNET  includes  use  of  unlimited  satellite  time.  When  selecting  a  system  for 
use,  the  satellite  costs  per  hour  must  be  considered. 
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During  this  project,  TNET  was  underutilized.  When  the  special  topics  courses  were 
presented,  the  system  was  only  being  used  three  or  four  days  a  month  for  delivery  of  instruction, 
yet  the  project  was  charged  for  an  entire  month  of  use.  Earlier  in  the  project,  even  though  the 
courses  were  ten  days  long,  the  system  was  being  used  for  instructor  practice  and  to  field  test 
the  graphics  and  some  of  the  instructional  activities.  Underutilization  of  expensive  technology 
such  as  TNET  greatly  increases  the  cost  of  course  delivery.  For  VTT  to  be  cost  effective,  TNET 
should  be  fully  used. 

All  VTT  courses  require  extensive  course  design  and  development;  the  FTP  courses  were 
no  exception.  These  design  and  development  costs  are  expensive.  Presumably,  this  expenditure 
resulted  in  excellent  instruction.  Again,  no  monetary  value  can  be  placed  on  superior  training. 
The  value  of  excellent  training  at  a  distance  versus  unavailable  training  at  a  resident  school  must 
be  considered  when  choosing  to  adopt  VTT. 

Four  TNET  systems  were  rented  out  of  project  funds  for  a  period  of  six  months.  When 
these  costs  were  compared  to  traditional  instruction  it  was  difficult  to  achieve  a  cost  savings 
unless  the  same  course  was  presented  multiple  times.  Because  the  design  and  development  costs 
are  one-time  expenses  and  are  not  incurred  each  time  the  course  is  presented,  delivering  the  same 
course  more  than  once  will  always  decrease  its  overall  cost. 

While  students  who  took  the  FTP  courses  wanted  college  credit,  and  the  military 
encourages  students  to  receive  credit  (Watt,  1988),  providing  credit  increased  the  costs  of 
delivering  the  courses  at  a  community  college.  Costs  associated  with  student  fees  and  credit 
accounted  for  between  1.26%  (TQL)  to  10.45%  (71L10)  of  the  total  remote  site  operating  costs. 
Thus,  while  the  costs  of  resident  training  were  considerably  less  for  four  of  the  five  courses, 
students  in  resident  training  do  not  have  the  benefit  of  receiving  credit.  Therefore,  a  cost 
comparison  of  VTT  delivered  at  the  community  colleges  and  at  the  resident  schools  does  not  take 
into  account  the  non-monetary  value  of  receiving  college  credit  for  military  training. 

Another  issue  arises  as  to  whether  or  not  the  costs  of  implementing  VTT  would  be  the 
same  as  for  the  FTP  if  the  military  had  an  on-going  VTT  program.  Administration  costs  (e.g., 
travel,  coordination,  etc.)  and  remote  costs  would  be  reduced  thus  making  VTT  a  more  viable 
cost-effective  training  option  and  one  to  be  considered  carefully. 

When  comparing  resident  costs  to  VTT  delivery,  an  issue  to  consider  is  how  to  get 
distance  delivery  at  a  competitive  cost  so  that  soldiers  have  the  option  of  taking  more  courses 
that  are  both  appropriate  for  VTT  and  that  might  not  otherwise  be  available  to  students.  While 
it  is  unlikely  that  distance  delivery  can  be  done  at  a  fraction  of  the  costs  of  resident  training, 
other  factors  such  as  access  and  college  credit  can  increase  the  value  of  VTT  for  the  military. 
Reasonable  differences  in  the  costs  of  the  two  options  may  be  more  acceptable  in  light  of  the 
importance  of  these  factors. 
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SUMMARY  AND  CONCLUSIONS  OF  THE  FLORIDA  TELETRAINING  PROJECT 

The  purpose  of  the  Florida  Teletraining  Project  was  to  assess  the  feasibility  of  using  two- 
year  community  colleges  to  offer  military  programs  of  instruction  that  addressed  specific  military 
duties  and  content.  The  courses  were  designed,  developed,  and  delivered  by  faculty  at  FCCJ  in 
conjunction  with  military  SMEs  from  two  Florida  USARF  schools.  DITRA  was  the  government 
agency  in  charge  of  program  management.  1ST  was  the  prime  contractor  and  provided  technical 
assistance  to  FCCJ. 

Five  courses  were  delivered  over  TNET,  the  Army’s  teletraining  network.  Three  of  the 
courses  were  U.S.  Army  Reserve  Component  Configured  Courseware  (RC3):  71L10, 

Administrative  Specialist;  76Y10,  Unit  Supply  Specialist;  and  9SB10,  Basic  Military  Police. 
These  courses  were  delivered  once  each  to  Army  National  Guard  and  Army  Reserve  soldiers  who 
were  seeking  to  be  reclassified  in  these  MOSs.  Two  of  the  courses  were  Navy  special  topics 
courses:  Handling  Hazardous  Waste-Activity  Level  (HazWaste)  and  Total  Quality  Leadership 
(TQL).  These  two  courses  addressed  joint  service  needs  and  were  made  available  to  members 
of  interested  services  and  components,  e.g.,  Army  National  Guard,  U.S.  Marines,  Air  National 
Guard,  the  U.S.  Navy,  and  the  U.S.  Coast  Guard.  TQL  and  HazWaste  were  offered  two  and 
three  times  respectively. 

The  courses  were  delivered  to  three  Florida  community  college  remote  sites:  SPJC,  VCC, 
and  a  classroom  at  FCCJ.  HazWaste  and  TQL  were  also  offered  at  two  out-of-state  sites  during 
the  final  administrations  of  these  courses:  Ft  Taylor  Hardin  in  Montgomery,  Alabama  and  Camp 
Fogarty  in  East  Greenwich,  Rhode  Island. 

Twenty-one  evaluation  objectives  were  identified  for  the  project  in  the  following 
categories:  technology,  instruction,  community  colleges,  and  cost.  Data  related  to  each  of  the 
objectives  were  collected  from:  (a)  students,  (b)  course  developers/VTT  instructors,  (c)  all 
remote  site  personnel  including  ICs,  POCs,  and  technicians,  (d)  administrative  project  staff  at 
FCCJ,  (e)  the  technical  and  production  teams  at  FCCJ,  and  (f)  military  personnel  including 
SMEs,  on-camera  military  instructors,  and  military  remote  site  personnel. 

Each  objective  was  addressed  separately  in  the  final  report  and  a  summary  of  the  data  was 
presented.  In  addition,  each  category  of  objectives,  i.e.,  technology,  instruction,  community 
colleges,  and  cost,  was  summarized.  Issues  related  to  the  objectives  were  identified.  In  general, 
the  TNET  system  was  reliable,  the  instruction  was  effective,  and  the  community  colleges  were 
able  to  present  high  quality  instruction.  However,  in  this  project  VTT  was  more  expensive  to 
implement  than  resident  training. 
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The  following  list  is  a  brief  overview  of  the  major  findings  related  to  the  FTP. 

Technology: 


•  TNET  was  99%  reliable;  students  and  instructional  personnel  rated  the  quality  of 
the  system  high. 

•  Students  and  ICs  indicated  that  they  preferred  a  VTT  approach  to  traditional 
training  at  a  military  facility. 

•  The  courses  selected  for  VTT  presentation  were  generally  suitable  for  the 
technology  although  some  modifications  were  made  (e.g.,  off-line  activities)  to 
accommodate  the  technology  and  the  course  content. 

•  Even  though  training  was  provided,  the  instructional  personnel  felt  that  more 
training  and  practice  was  needed  on  TNET. 

Instruction: 

•  All  students  passed  the  stated  learning  objectives,  and  over  90%  of  all  students  in 
the  MOS  courses  passed  the  performance  tests  on  the  first  attempt. 

•  No  comparisons  could  be  made  between  the  performance  records  of  students  in 
the  VTT  courses  and  those  in  other  training  options. 

•  Students  rated  the  learning  methods  and  activities,  including  the  interactivity 
provided  in  the  VTT  courses,  to  be  effective.  These  learning  methods  stimulated 
student  interest  and  the  students  were  generally  motivated  by  the  VTT  instruction. 

•  No  student  demographic  variables  predicted  success  or  lack  of  it  for  military 
students  and  hence  one  can  expect  that  VTT  is  an  acceptable  approach  for  the 
general  military  population. 

•  Students  rated  all  the  instructional  personnel,  VTT  instructors,  MIAs,  ICs,  and 
MSCs,  as  effective.  A  question  was  whether  or  not  civilians  can  provide  quality 
instruction  to  military  students  due  to  their  lack  of  military  training.  Some 
students  felt  that  a  military  instructor  should  have  presented  the  instruction, 
however,  all  students  passed  the  learning  objectives  for  all  five  courses. 

Community  Colleges 

•  FCCJ  had  the  technical  and  instructional  capabilities  to  implement  VTT 

instruction.  The  technics  %  TV  studio,  and  production  facilities  (one  of  the 

best  among  community  c  s  in  Florida)  were  excellent. 
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•  Community  college  faculty  are  professional  educators.  While  the  faculty  at  FCCJ 
lacked  instructional  design  and  military  training  expertise,  they  were  able  to  design 
and  present  high  quality  instruction  given  specific  training. 

•  Staff  training  was  presented  to  all  instructional  and  technical  personnel.  This 
training  was  judged  to  be  of  high  quality  by  the  staff  and  they  indicated  that  it 
enabled  them  to  perform  their  roles  and  responsibilities. 

•  The  course  developers/VTT  instructors  had  the  most  extensive  training  of  all  the 
instructional  personnel.  This  included  instruction  in  VTT  presentation, 
instructional  design,  and  military  training. 

•  The  community  colleges  were  able  to  grant  academic  and  continuing  education 
credit  for  all  courses.  Although  not  all  students  were  interested  in  receiving 
credit,  those  that  were,  indicated  that  college  credit  was  a  definite  advantage  to 
receiving  military  training  at  a  community  college. 

•  The  three  community  colleges  were  organized  to  work  together  through 
subcontracts;  this  proved  to  be  a  very  effective  mechanism. 

•  There  was  a  complex  web  of  community  college,  university,  government,  and 
military  organizations  that  had  to  work  together  to  implement  this  project.  Formal 
contracts,  MOUs,  IPRs,  and  a  number  of  informal  agreements  enabled  the  groups 
to  work  together  effectively. 


Costs: 


•  With  the  exception  of  the  HazWaste  course,  the  VTT  courses  in  the  FTP  were 
me  .  pensive  when  compared  to  resident  training.  The  HazWaste  course  was 
cos  f  '«ctive  because  there  were  a  large  number  of  students  (N=116)  in  the 
course  and  because  of  travel  and  per  diem  savings. 

•  The  costs  associated  with  providing  college  credit  (in-state  and  out-of-state)  and 
the  lease  of  the  four  TNET  systems  greatly  increased  the  costs  of  the  VTT 
courses. 

•  The  VTT  courses  become  more  cost  effective  if:  (a)  the  same  course  is  presented 
more  than  once  (because  design  and  development  costs  are  paid  only  once),  and 
(b)  different  courses  are  presented  in  the  same  month,  (because  each  course  would 
be  charged  only  for  the  days  the  course  is  implemented  rather  than  paying  the 
TNET  costs  for  the  entire  month). 
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Lessons  Learned 


•  The  courses  developed  for  this  project  were  used  as  a  component  of  the  evaluation 
of  the  pilot  test.  Decisions  made  for  the  purposes  of  the  project  may  not  have 
been  had  this  not  been  a  pilot  test.  For  example,  students  were  assigned  to 
a  given  remote  site,  not  necessarily  the  one  closest  to  their  home,  to  insure  that 
there  were  sufficient  number  of  students  for  the  evaluation.  However,  under 
regular  instructional  circumstances,  students  should  be  assigned  to  the  nearest 
remote  site. 

•  The  choice  of  technologies,  in  this  case  TNET,  influenced  the  design  of 
instruction  because  of  the  video  transmission  rate.  The  FTP  courses  are  not 
directly  adaptable  to  other  VTT  systems.  They  would  have  to  be  modified  to  be 
used  effectively. 

•  TNET  transmission  is  influenced  by  inclement  weather.  Contingency  plans  should 
be  devised  that  can  be  used  if  instruction  is  interrupted  for  any  reason. 

•  Origination  site  technicians  must  establish  a  working  relationship  with  NCC.  This 
allows  problems  to  be  solved  in  a  timely  and  efficient  matter. 

•  Origination  and  remote  site  technicians  are  invaluable  resources.  Origination  site 
technicians  should  be  knowledgeable  in  various  communication  technologies. 

•  VTT  must  be  considered  in  the  context  of  other  training  options.  Courses  should 
be  selected  for  VTT  delivery  that  are  short  (one  to  two  weeks  in  length),  are 
primarily  cognitive,  and  have  high  demand  or  throughput. 

•  Colleges  must  provide  adequate  facilities  for  VTT  instruction,  e.g.,  classroom 
space,  facilities,  etc. 

•  While  the  TNET  equipment  is  not  difficult  to  use,  some  personnel  are  intimidated 
by  any  new  technology.  Care  must  be  taken  to  provide  sufficient  practice  and 
technical  support  to  such  personnel  when  the  technology  is  in  use. 

•  Military  POIs  and  syllabi  can  be  successfully  reconfigured  for  VTT  instruction. 

•  The  results  of  the  data  from  the  PTs  may  assist  proponent  schools  to  evaluate  their 
curricula. 

•  It  took  longer  to  present  RC3  courses  by  VTT  than  to  present  platform  instruction 
because  of  planned  interaction  and  the  administration  of  on-site  PEs.  If  the 
military  adopts  VTT,  the  overall  course  length  will  increase  in  order  to  present  the 
same  content. 
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•  Courses  that  are  taught  in  day  long  sessions  (e.g.,  8  hours/day)  need  to  be 
designed  so  that  students  have  a  break  from  viewing  the  TNET  monitor.  Off-line 
activities,  including  activities  that  are  interactive,  are  one  way  to  accomplish  this 
goal. 

•  Students  who  were  more  highly  educated  or  held  a  higher  military  grade  or  rank 
were  more  critical  than  others  of  the  word  picture  concept  that  was  used  in  this 
project.  Other  ways  of  involving  these  students  in  the  instructional  process  needs 
to  be  explored. 

•  Careful  attention  must  be  paid  to  designing  activities  that  allow  students  to  interact 
with  each  other  on  the  network. 

•  The  different  cultures  of  the  military  and  the  community  college,  in  addition  to 
the  roles  and  responsibilities  assigned  to  each  group,  require  coordination  by 
individuals  who  can  link  the  two  communities. 

•  Military  and  community  college  coordination  with  regard  to  presenting  VTT 
instruction  should  begin  early  during  any  project. 

•  If  community  colleges  and  the  military  are  going  to  work  together  to  provide 
effective  and  cost  efficient  training  to  military  students,  using  both  ICs  and  MSCs 
at  the  remote  sites  is  unnecessary.  The  issues  surrounding  how  to  grant  college 
credit  and  how  to  certify  military  students  with  either  the  IC  or  MSC  must  be 
resolved. 

•  If  VTT  is  adopted  by  any  military  training  organization,  issues  about  the  roles  of 
existing  personnel  will  arise.  Resident  course  instructors,  like  community  college 
faculty,  can  be  learn  to  present  VTT  courses  given  appropriate  staff  training. 

•  Although  the  military  can  be  taught  to  deliver  instruction  via  VTT,  military 
instructors  felt  apprehension  about  the  loss  of  positions  and  about  assuming  new 
roles  if  VTT  is  widely  adopted. 

•  Course  design,  development,  and  delivery  required  a  team  approach  because  the 
community  college  faculty  who  taught  the  courses  did  not  have  sufficient  military 
background.  A  military  SME  assisted  the  community  college  faculty  in  design, 
development,  and  implementation.  This  doubling  of  resources  was  neither  time 
nor  cost  effective. 

•  The  quality  of  the  POIs  and  syllabi  received  from  the  military  directly  effects  the 
time  and  the  costs  for  reconfiguring  military  courses  for  VTT. 

•  The  larger  the  population  that  can  be  served  using  VTT,  the  greater  the  cost 
savings.  An  on-going  VTT  program  would  be  more  cost  effective. 
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•  It  is  nearly  impossible  to  compare  VTT  and  resident  training  without  complete 
data  on  the  cost  of  resident  training.  This  would  include  the  costs  for  design, 
development  and  delivery  of  the  courses,  and  for  the  base  operating  costs. 

•  If  the  goal  of  a  project  such  as  the  FTP  is  to  compare  traditional  and  mediated 
instruction,  all  associated  costs  for  all  training  options  must  be  made  available. 
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Interactive  Study  Guide 
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SAT 

Systems  Approach  to  Training 

SMART 

System  for  Managing  Asynchronous  Remote  Training 

SME 

Subject  Matter  Experts 

SPJC 
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FTP  Evaluation  Instruments 


Form 

Data 

Source 

Data 

Collection  Schedule 

Course 

A 

Student  Background  Questionnaire 

Student 

Prior  to  Inst 

B 

MOS  Related  Questionnaire 

Student 

Prior  to  Inst 

MOS  only 

C-l 

Pretest 

Student 

Prior  to  Inst 

C-2 

Posttest 

Student 

After  Inst 

D 

Entry  Typing  Aptitude  Score  Sheet 

Student 

Prior  to  Inst 

71L10  only 

E 

IC  Daily  Course  Rating 

IC 

End  of  each  day 

F 

End-of-Course  Typing  Score  Sheet 

Student 

After  Course 

71L10  only 

G 

Student  Course  Evaluation 

Student 

Last  Event 

H 

Student  Rating:  Teaching  &  Delivery 

Student 

Last  Event 

I 

Conditions:  Effective  Crs  Delivery 

Evaluator 

End  of  Course 

I-A 

Out-of-State  Site  Information 

Evaluator 

End  of  Course 

HAZ/TQL  only 

J 

Formative/Intensive  Site  Evaluation 

Student 

1-2-3  Day 

71L10  only 

J-A 

Evaluator  Observation 

Evaluator 

During  Inst 

K 

Comm  College  Admin  POC  Interview 

Admin 

End  of  Course 

L 

Student  Interview  Form 

Student 

End  of  Course 

MOS  only 

GHL 

Course  Critique 

Student 

End  of  Course 

HAZ/TQL  only 

M 

MIA  Interview  Form 

MIA 

End  of  Course 

MOS  only 

M-l 

Course  Design  &  Development 

VTT  Inst 

End  of  Course 

HAZ/TQL  only 

M-2 

MIA  Implementation 

MIA 

End  of  Course 

HAZ/TQL  only 

N 

VTT  Instructor  Interview  Form 

VTT  Inst 

End  of  Course 

MOS  only 

O 

Comm  College  IC  Interview 

IC 

End  of  Course 

MOS  only 

P 

MSC  Interview  Form 

MSC 

End  of  Course 

MOS  only 

Q 

Comm  College  IC  Questionnaire 

IC 

End  of  Course 

QA 

MSC  Questionnaire 

MSC 

End  of  Course 

95B10  only 

*R 

Unit  Level  Military  Supervisor 

Admin 

End  of  Course 

MOS  only 

*s 

Interview  Unit  Training  NCO 

Admin 

End  of  Course 

MOS  only 

T 

TNET  Log 

Techn 

Daily 

V 

Performance  Test  Record 

VTT 

Daily 

MOS  only 

w 

Student  Aptitude  Area  Scores 

Student 

End  of  Course 

MOS  only 

X 

Cost  Data 

Evaluator 

End  of  Course 

MOS  only 

Y 

Background  Data  on  Civilians 

Students 

Prior  to  Inst 

MOS  only 

Z 

Background  Data  on  Military 

Students 

Prior  to  Inst 

MOS  only 

Military  Forms 

FBH30244  USARF  Go-No-Go  Form  MSC 

Daily 

MOS  only 

FORSCOM  USARF  School  Attendance 

MSC 

Daily 

MOS  only 

270-R 

2AA  Form  USAR  School  Class  Status 

MSC 

Daily 

MOS  only 

26-R 

81st 

All  Course  Training  Progress 

MSC 

Daily 

MOS  only 

ARCOMIOO 

2AA  Form  USAR  School  Visitors  Register 

MSC 

Daily 

MOS  only 

29-R 

2AA  Form  USARF  School  Classroom  Eva! 

MSC 

Daily 

MOS  only 

54-l-R 


Instructional  Coordinator  IC 

Military  Instructional  Assistant  MIA 

Military  Site  Coordinator  MSC 

Student  S 

Video  Teletraining  VTT 


*  Not  Collected  -  DITRA  Directive 
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Appendix  B 

Description  of  Evaluation  Forms 

•  Forms  A,  B,  Y,  and  Z  were  used  to  collect  demographic  and  background 
information  from  the  students.  Form  W  was  used  to  record  ASVAB  scores  for 
students  in  the  MOS  courses. 

•  Forms  Cl,  C2,  D,  and  F  were  the  pre  and  post  test  instruments  used  in  each  of  the 
five  courses.  These  instruments  are  described  in  Objective  Bl. 

•  Form  V  was  used  to  record  the  students’  scores  on  the  performance  tests  for  the 
MOS  courses. 

•  Forms  G,  H,  L,  and  GHL  were  used  to  collect  course  rating  data  from  the 
students.  Both  open-ended  and  closed  response  items  were  asked.  Students  stated 
their  perceptions  about  the  adequacy  of  courses,  the  equipment,  the  instructors, 
etc.  Form  GHL  was  a  composite  of  Forms  G,  H,  and  L  and  was  given  to  the 
HazWaste  and  TQL  students  because  of  time  constraints. 

•  Forms  M,  Ml,  M2,  and  N  were  used  to  collect  course  preference  data  from  the 
VTT  nstructors  and  the  MIAs.  A  wide  range  of  questions  concerning  their 
perceptions  about  the  project,  the  course  development  and  course  delivery 
processes,  the  students,  and  TNET  were  asked. 

•  Forms  K,  O,  P,  Q,  and  QA  were  used  to  collect  data  from  the  civilian  and  remote 
site  personnel  regarding  their  perceptions  about  course  delivery,  TNET,  the 
students,  etc. 

•  Form  E  was  the  instrument  used  to  receive  feedback  from  the  remote  sites  about 
each  instructional  day.  These  forms  were  used  as  the  basis  for  the  AARs 
conducted  at  the  conclusion  of  each  day. 

•  Form  T  was  used  to  record  information  about  the  TNET  equipment  (i.e.,  when  the 
equipment  was  turned  on  and  off,  and  any  problems  that  occurred). 

•  Forms  I  and  IA  were  checklists  used  by  the  project  evaluators  to  determine 
whether  or  not  each  site  had  all  the  materials  and  equipment  required  for  each 
course  on  the  first  day  of  delivery.  For  example,  evaluators  checked  to  see  that 
all  course  materials  were  available  and  that  the  TNET  equipment  was  operative. 

•  Form  J  was  used  at  FCCJ2  only.  This  site  was  designated  as  the  formative 
evaluation  site.  On  the  first  several  days  of  each  of  the  MOS  courses,  students 
provided  feedback  about  the  adequacy  of  the  course,  and  one  evaluator  rated  each 
course.  This  data  was  then  used  to  make  immediate  corrections  to  the  courses. 
The  data  was  also  used  to  influence  the  development  of  the  other  courses. 
Therefore,  these  data  were  not  used  to  evaluate  the  project,  but  were  used  as  a 
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component  of  course  development.  This  approach  is  described  in  the 
Reconfiguration  document. 

•  Form  J-A  was  used  by  the  FTP  evaluators  to  record  interaction  patterns  during 
each  of  the  MOS  courses. 

•  Form  X  was  used  to  collect  the  cost  data. 

•  The  military  forms  used  were  standard  Army  forms.  These  were  used  during  each 
of  the  MOS  courses  so  that  the  students  could  be  certified. 
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165 


Appendix  C 

Network  Guidelines  for  the  Florida  Teletraining  Project 


•  Equipment  was  used  only  by  technicians,  instructors,  and  others  trained  in  its  use  by 
certified  ATSC/TNET  personnel. 

•  Use  was  scheduled  in  15  minute  blocks.  Prior  to  each  use,  30  minutes  was  reserved  for 
pre-use  testing. 

•  A  log  of  use,  maintenance  and  down  time  was  maintained  for  the  duration  of  the  project 
using  log  sheets  provided  by  the  Project. 

•  Visitors  were  allowed  on  site  to  observe  teletraining  in  progress,  but  no  interruptions  or 
disruption  of  instruction  by  visitors  were  permitted  for  any  reason. 

•  Security  was  maintained  for  the  classrooms  designated  for  instruction.  Each  room  was 
locked  when  not  in  use  for  this  project. 

•  All  equipment  was  secured  during  instruction,  during  instructional  breaks  and  between 
instructional  periods. 

•  Designated  TNET  classrooms  could  be  used  by  approved  military  units  and  community 
college  instructors  for  other  than  FTP  instruction  if  the  following  criteria  were  met: 

•  Intended  use  was  acceptable  to  the  administration  at  the  individual  receive  sites 

•  Requests  had  to  be  submitted  through  FCCJ  at  least  five  (5)  working  days  in 
advance  of  intended  use.  FCCJ  scheduled  the  conferences  with  the  NCC. 

•  Use  could  not  interfere  with  any  training,  demonstrations,  instructional  or 
evaluation  activities  that  were  part  of  the  FTP 

•  Use  had  to  be  acceptable  in  terms  of  time  and  availability  of  classroom  space, 
equipment  and  satellite  time 

•  Requests  for  time  and  date  changes  had  to  be  made  through  the  Network  Management 
at  FCCJ. 

•  A  TNET  trained  technician  had  to  be  on  hand  if  the  equipment  was  used. 

•  Rules  and  regulations  as  established  for  equipment  use  for  FTP  had  to  be  followed. 
These  included  classroom  protocol  and  control  of  the  audio  at  the  remote  sites  were 
located  in  the  Instructor  Guide,  Instructional  Coordinator  Guide  and  the  Interactive  Study 
Guide  developed  for  each  course. 
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Table  Bl.l 


71L10:  Performance  Test 

(N  =  33) 


Test 

Part 

Initial 
%  GOstN) 

X  (SD) 

Initial  GOs 

Retest 
%  GOs  (N) 

%  Final  GOs 

Resupply 

I 

75.8%  (25) 

93.20(11.08) 

100%  (8) 

100% 

n 

100%  (33) 

97.59  (5.00) 

N/A  (0) 

100% 

Memo 

i 

87.9%  (29) 

88.83  (8.89) 

100%  (4) 

100% 

SP  Memo 

i 

97.0%  (32) 

87.44  (11.03) 

100%  (1) 

100% 

n 

97.0%  (32) 

97.56  (3.69) 

100%  (1) 

100% 

Endorsement 

i 

97.0%  (32) 

91.37  (9.26) 

100%  (1) 

100% 

h 

93.9%  (30) 

95.94  (4.75) 

100%  (3) 

100% 

Letter 

i 

97.0%  (32) 

93.23  (5.31) 

100%  (1) 

100% 

Assemble  Cones. 

i 

51.5%  (17) 

84.71  (11.79) 

100%  (16) 

100% 

Joint  Message 

i 

93.9%  (31) 

88.87  (12.83) 

100%  (2) 

100% 

R/T  Classified 

i 

90.9%  (30) 

99.35  (2.50) 

100%  (3) 

100% 

n 

93.9%  (30) 

94.84  (9.62) 

100%  (3) 

100% 

Route  I/O 

i 

100%  (33) 

88.18  (14.89) 

N/A  (0) 

100% 

n 

97.0%  (32) 

89.69  (14.48) 

100%  (1) 

100% 

m 

93.9%  (31) 

100.0  (.00) 

100%  (2) 

100% 

IV 

63.6%  (21) 

80.00  (14.49) 

100%  (12) 

100% 

V 

93.9%  (31) 

94.19(12.05) 

100%  (2) 

100% 

Files 

i 

87.9%  (29) 

86.90  (10.04) 

100%  (4) 

100% 

n 

63.6%  (21) 

100.0  (.00) 

100%  (12) 

100% 

in 

84.8%  (28) 

100.0  (.00) 

100%  (5) 

100% 

Average 

90.03% 

100%  (4) 

100% 

Typing 

36%  (12) 

29.83  (5.29) 

82%  (27) 

36.89  (8.98) 

Passed  Pretest 

Passed  Posttest 
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Table  B1.2 


76Y10:  Percentage  Passing  Proficiency  Tests 

(N  =  40) 


UNIT 

Initial 
%  GOs  (N) 

X  (SD) 

Retest 

D3.9  ARMS 

97.5%  (39) 

93.51  (6.44) 

100%  (1) 

D4.9  Status  files/requests 

92.5%  (37) 

81.11  (8.91) 

100%  (3) 

D5.9  Hand  receipts  &  issuance 

100%  (40) 

90.65  (5.36) 

N/A 

D6.9  Relief  from  responsibility 

100%  (40) 

90.12  (4.13) 

N/A 

E2.9  Personal  clothing  requests 

100%  (40) 

96.90  (2.81) 

N/A 

E3.9  Organizational  clothing  & 
individual  equipment 

97.5%  (39) 

90.23  (8.06) 

100%  (1) 

Total 

100% 

100% 

100% 

100% 

100% 

100% 


Average 


97.99% 


Table  B1J 


UNIT 
ZH  200 

LH  200 
LH  206 


LH204 

LH  210 
LH  210 

LH  400 


95BI0:  Performance  Test  Results 

(N  =  26) 


Initial 

%  GOs  (N)  X  (SD)  %  Final  GOs 


Military  Law 

Lawful  Apprehension 

77.8%  (21) 

94.18  (6.57) 

100% 

Search  &  Seizure 

100%  (26) 

95.74  (5.58) 

100% 

Miranda  Rights 

92.6%  (25) 

94.04  (6.86) 

100% 

Use  of  Force 

92.6%  (25) 

91.11  (10.13) 

100% 

Apprehension 

Apprehend  a  Subject 

100%  (26) 

98.81  (3.88) 

100% 

Select  Search 

100%  (26) 

99.26  (2.67) 

100% 

Prone  Search 

77.8%  (21) 

97.62  (4.18) 

100%  (6) 

Stand-up  Search 

83.3%  (24) 

96.24  (5.38) 

100% 

Wall  Search 

92.6%  (25) 

96.74  (4.10) 

100% 

Pat  Down  Search 

92.6%  (25) 

99.88  (0.59) 

100% 

Reports  and  Forms 

DA  Form  2823 

74.1%  (20) 

97.04  (5.16) 

100% 

DA  Form  3975 

55.6%  (15) 

95.19  (6.36) 

100% 

Gather  Information 

96.3%  (26) 

93.08  (9.28) 

100% 

Conduct  Interviews 

96.3%  (26) 

95.85  (6.77) 

100% 

Patrol  Incidents 

Traffic 

84.6%  (22) 

94.08  (6.06) 

100%  (1) 

Accident 

92.3%  (24) 

94.72  (6.78) 

100% 

DUI 

92.3%  (24) 

95.29  (6.29) 

100% 

Transport 

96.2%  (25) 

93.24  (7.26) 

100% 

Direct 

100%  (26) 

100.0  (0.00) 

100% 

Bomb 

100%  (26) 

86.92  (8.84) 

100% 

Alarms 

100%  (26) 

95.00  (6.84) 

100% 

Hostage 

100%  (26) 

90.80  (7.86) 

100% 

Domestic 

100%  (26) 

93.80  (4.91) 

100% 

Protect 

100%  (26) 

92.50  (8.28) 

100% 
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Table  B1 3 


VNIT 

LH  208 

Identify  Evidence 
Section  1 

Section  2 

Section  3 

Section  4 

LH  200 

Child  Abuse 

SH  202 

Nuclear  Security 
Escort 

Patrol 

Enforce 

Duress 

LH  202 

Self  Defense 

Phase  I 

Phase  II 

(Continued) 


Initial 

%  GOs  (N) 

X(S_D) 

96.2%  (25) 

98.00  (5.00) 

96.2%  (25) 

92.27  (9.08) 

88.5%  (23) 

100.0  (0.00) 

100%  (26) 

100.0  (0.00) 

100%  (26) 

93.85  (9.41) 

100%  (26) 

92.12  (10.21) 

100%  (26) 

93.85  (9.41) 

100%  (26) 

96.92  (7.36) 

100%  (26) 

89.42  (10.03) 

100%  (26) 

100.0  (0.00) 

100%  (26) 

96.92  (5.27) 

%  Final  GOs 


100% 

100% 

100% 

100% 

100% 


100% 

100% 

100% 

100% 


100% 

100% 


Average 

*(x)  =  Number  of  Missing  Cases 


94.5% 


Table  B1.4 


71L10:  Student  Self-Assessment  of  Proficiency 
%  Students  (N*) 


Task 

Have  Performed  Task 

Know  How  to  Perform  Task 

YES 

NO 

YES 

NO 

Resupply 

15.2%  (5) 

72.7%  (24) 

9.1%  (3) 

78.8%  (26) 

SP  Memo 

15.2%  (5) 

72.7%  (24) 

6.1%  (2) 

81.8%  (27) 

Letter 

54.5%  (18) 

33.3%  (11) 

45.5%  (15) 

42.4%  (14) 

Joint  Message 

15.2%  (5) 

72.7%  (23) 

3.0%  (1) 

84.8%  (28) 

R/T  Classified 

6.1%  (2) 

81.8%  (27) 

0.0%  (0) 

87.9%  (29) 

Route  VO 

18.2%  (6) 

69.7%  (23) 

6.1%  (2) 

81.8%  (27) 

Missing  data  =  at  least  4  students  for  all  items 
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Table  B1.5 


76Y10:  Average  Scores  on  Self 'Report  Questionnaire 

5  Point  Scale1 
(N  =  40) 


UNIT 

PRE 

POST 

D2.9  File  Procedures 

3.2 

4.7 

D3.9  ARMS 

2.5 

4.6 

D4.9  States  files/requests 

2.4 

4.5 

DS.9  Hand  receipts  &  issuance 

2.7 

4.7 

D6.9  Relief  from  responsibility 

2.1 

4.6 

E2.9  Personal  clothing  requests 

2.2 

4.8 

E3.9  Organizational  clothing  & 
individual  equipment 

2.6 

4.6 

Average 

2.5 

4.6 

'Note:  5  =  1  can  perform  the  task  proficiently  and  to  Army  specifications. 

4  =  I  can  perform  the  task  adequately,  but  I  need  a  reminder  or  assistance  to  help  me  perform  the  task  proficiently. 

3  s  I  can  perform  the  task  to  some  extent,  but  I  am  not  proficient  even  with  some  assistance. 

2  =  1  cannot  perform  the  task,  but  1  am  familiar  with  it. 

1  =  I  cannot  perform  the  task  and  I  am  not  familiar  with  it. 
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Table  B1.6 

95B10:  Average  Scores  on  Self-Report  Questionnaire 

5  Point  Scale1 
(N  =  26) 


UNIT 

PRE 

POST 

LH  200 

1PC  Skills  (IMPACT) 

2.1 

4.5 

LH  200 

Military  Law 

1.9 

4.7 

LH  200 

Use  of  Force 

2.7 

4.9 

LH  206 

Apprehension 

2.2 

4.7 

LH  204 

Prepare  Military  Police 

Reports  &  Forms 

1.3 

4.7 

LH  210 

Gather  Information/ 

Conduct  Interviews 

1.7 

4.8 

LH  400 

Perform  Patrol  Duties 

1.9 

4.7 

LH  208 

Identify  (Collect)  and 

Process  Evidence 

1.7 

4.7 

LH  200 

Child  Abuse 

2.6 

4.7 

SH  202 

Nuclear  Physical  Security 

2.0 

4.8 

LH  202 

Self  Defense 

2.0 

4.8 

Average 

2.0 

4.7 

'Note:  5  = 

4  = 
3  = 
2  = 
I  = 


1  can  perform  the  task  proficiently  and  to  Army  specifications. 

I  can  perform  the  task  adequately,  but  I  need  a  reminder  or  assistance  to  help  me  perform  the  task  proficiently. 
1  can  perform  the  task  to  some  extent,  but  I  am  not  proficient  even  with  some  assistance. 

I  cannot  perform  the  task,  but  1  am  familiar  with  it. 

I  cannot  perform  the  task  and  1  am  not  familiar  with  it. 
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Table  B1.7 


TQL:  Average  Scores  on  Self-Report  Questionnaire 

5  Point  Scale1 
(N  =  59) 


MODULE 

PRE 

Module  1:  Background 

2.3 

Module  2:  Red  Bead 

No  Data  Collected 

Module  3:  Basic  Principles 

u 

Module  4:  Method/Tools 

1.9 

Module  5:  Implementation 

1.3 

'Note:  5  = 

4  at 
3  = 
2  = 
1  = 


I  can  perform  the  task  proficiently. 

1  can  perform  the  task  adequately,  but  I  need  a  reminder  or  assistance  to  help  me  perform  the  task  proficiently. 
I  can  perform  the  task  to  some  extent,  but  1  am  not  proficient  even  with  seme  assistance. 

I  cannot  perform  the  task,  but  I  am  familiar  with  it. 

I  cannot  perform  the  task  and  I  am  not  familiar  with  it. 
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Appendix  F 

Tables  from  Objective  B3 


181 


Table  B3.1 


Student  Ratings  of  Instructional  Methods 
(5  Point  Scale,  5  =  Highest) 

X  (SD) 

(%  Ranking  3,4,5) 

(N) 


ITEM 

71L10 

76Y10 

95B10 

HAZ 

Word  graphics 

3.65  (1.14) 
(75.7%) 

(31) 

3.82  (1.23) 
(82.5%) 

(38) 

3.44  (1.04) 
(76.9%) 

(25) 

4.33  (0.77) 
(96.4%) 
(110) 

Value  of 

Practical  exercises 

4.18  (0.98) 
(90.9%) 

(33) 

3.44  (1.19) 
(95.0%) 

(32) 

4.04  (0.89) 
(92.3%) 

(25) 

4.26  (0.82) 

(95.5%) 

(HI) 

Instructional  games 

2.61  (1.45) 
(42.4%) 

(28) 

3.44  (1.19) 
(62.5%) 

(32) 

3.46  (1.25) 
(69.2%) 

(24) 

NA 

Remediation  helpful 

4.13  (0.88) 
(87.9%) 

(31) 

3.68  (1.02) 
(80.0%) 

(40) 

4.05  (0.80) 
(80.8%) 

(21) 

NA 

Typing  effective1 

4.39  (1.03) 
(93.9%) 

(33) 

Microfiche  effective2 

3.82  (1.23) 
(80.0%) 

(40) 

MSC  directing  PEs3 

4.60  (0.58) 
(96.2%) 

(25) 

71L10  only 
76Y10  only 
95B10  only 


TOL 

4.43  (0.65) 

(97.9%) 

(47) 


4.15  (0.8  i) 

(95.8%) 

(47) 


NA 


NA 
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Table  B3.2 


Student  Ratings  of  Interactivity 
(5  Point  Scale,  5  =  Highest) 

X  (SD) 

(%  Ranking  3,4,5) 

(N) 


ITEM 

71L10 

76Y10 

95B10 

HAZ 

TOL 

Opportunities  to  ask 
questions 

4.09  (0.89) 
(93.9%) 

(32) 

4.03  (1.19) 
(85.0%) 

(40) 

4.31  (0.84) 
(100%) 

(26) 

4.18  (0.89) 

(95.5%) 

(HI) 

4.21  (0.77) 

(97.9%) 

(48) 

Opportunities  to  ask 
follow-up  questions 

4.00  (0.92) 
(93.9%) 

(32) 

4.00  (1.20) 
(85.0%) 

(40) 

4.23  (0.82) 
(100%) 

(26) 

NA 

NA 

Interaction  w/students 
on-site 

4.62  (0.61) 
(97.0%) 

(32) 

4.47  (0.82) 
(32.5%) 

(40) 

4.58  (0.50) 
(100%) 

(26) 

4.33  (0.74) 

(99.1%) 

(HI) 

4.11  (0.76) 

(97.9%) 

(47) 

Interaction  w/students 
on-network 

3.38  (1.48) 
(63.6%) 

(32) 

3.50(1.41) 

(75.0%) 

(40) 

3.08  (1.06) 
(73.1%) 

(26) 

N/A 

N/A 

Interaction  w/VTT 
instructor 

4.23  (0.88) 
(87.9%) 

(31) 

4.15  (1.08) 
(90.0%) 

(40) 

3.60(1.15) 

(84.6%) 

(25) 

4.33  (0.74) 
(97.3%) 
(111) 

4.23  (0.73) 

(95.8%) 

(47) 

Interaction  with 

instructional 

coordinator 

4.72  (0.52) 
(97.0%) 

(32) 

3.92  (1.38) 
(80.0%) 

(40) 

4.40  (1.00) 
(88.5%) 

(25) 

4.39  (0.66) 

(99.1%) 

(110) 

4.21  (0.91) 

(89.6%) 

(47) 

Interaction  w/milibry 
site  coordinator 

4.59  (0.71) 
(93.9%) 

(32) 

4.15  (1.25) 
(82.5%) 

(40) 

4.76  (0.44) 
(96.2%) 

(25) 

N/A 

N/A 
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Table  B3.3 

Student  Ratings  of  Course  Characteristics 
(5  Point  Scale,  5  =  Highest) 

X  (SD) 

(%  Ranking  3,4,5) 

(N) 


ITEM 

71L10 

76Y10 

95B10 

HAZ 

TQL 

General  course 
organization 

3.00  (1.22) 
(69.7%) 

(32) 

2.95  (1.18) 
(70.0%) 

(40) 

3.38  (1.06) 
(80.8%) 

(26) 

3.84  (0.93) 

(92.8%) 

(111) 

4.06  (0.95) 

(93.7%) 

(48) 

Quality  of  lesson 
presentations 

3.44  (1.01) 
(78.8%) 

(32) 

3.28  (1.13) 
(72.5%) 

(40) 

3.19  (1.17) 
(65.4%) 

(26) 

3.95  (0.85) 

(93.7%) 

(HO) 

4.00  (0.83) 

(97.9%) 

(48) 

Quality  of  printed 
materials 

3.88  (1.08) 
(75.8%) 

(33) 

3.55  (1.13) 
(85.0%) 

(40) 

3.65  (1.26) 
(80.8%) 

(26) 

4.17  (0.80) 
(98.2%) 
(111) 

4.19  (0.82) 

(95.8%) 

(48) 

Time  to  cover  topics 

2.69  (1.12) 
(51.5%) 

(32) 

3.68  (1.02) 
(85.0%) 

(40) 

3.42  (1.33) 
(76.9%) 

(26) 

3.24  (1.22) 
(75.7%) 
(111) 

3.13  (1.21) 

(66.7%) 

(48) 

Intro  to  VTT  course 

4.00  (1.06) 
(90.9%) 

(33) 

4.00  (1.20) 
(90.0%) 

(40) 

4.08  (0.95) 
(88.5%) 

(25) 

3.98  (0.85) 

(94.6%) 

(111) 

4.15  (0.77) 

(97.9%) 

(48) 

Intro  to  TNET  equipment 

4.06  (0.83) 
(97.0%) 

(33) 

3.55  (1  18) 
(90.0%) 

(40) 

4.08  (0.86) 
(92.3%) 

(25) 

4.01  (0.83) 

(96.4%) 

(111) 

4.08  (0.87) 

(95.8%) 

(48) 

Overall  course 
quality 

3.59  (0.98) 
(32) 

3.55  (1.18) 
(40) 

3.19  (1.10) 
(26) 

4.04  (0.92) 
(Ill) 

4.10  (0.81) 
(48) 

Pacing  of  course 
(3=about  right) 

3.48  (1.34) 
(69.7%) 

(31) 

2.68  (0.91) 
(60.0%) 

(34) 

2.04  (1.06) 
(34.6%) 

(25) 

2.94  (0.86) 

(75.7%) 

(HO) 

3.11  (0.87) 

(75.0%) 

(47) 
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Table  B3.4 


Instructional  Coordinator  Rankings  of  Course 
(5  Point  Scale,  5  =  Highest) 

X 

<N) 


ITEM 

71L10 

76Y10 

95B10 

Quality  of  VTT  lesson 

4.80 

4.00 

4.17 

presentations 

(30) 

(34) 

(39) 

Student  interaction 

4.87 

4.21 

4.21 

with  teacher 

(30) 

(34) 

(39) 

Instructor’s  use 

4.67 

3.98 

4.02 

graphics/charts 

(30) 

(34) 

(39) 

Wise  use  of  available 

4.70 

3.93 

3.69 

time 

(30) 

(34) 

(39) 

Classroom  use  of  exercises, 

4.79 

4.04 

4.14 

games,  activities 

(30) 

(34) 

(39) 
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Related  civilian 
occupation 


Table  B5.1 
(continued) 


Related  work 
experience  (civ) 


Table] 
(con  tin 


Table  B5 2 

Regression  of  Objective  Posttest  Scores  on  Student  Background  Characteristics 


Step  1 
MoColl 

CompSkl 

MilRes 

MilRes 

ASVAB-ST 

PMOS 

PMOS 

Step  2 

— 

ASVAB-CL 

— 

— 

MilRes 

R 

.65 

.35 

.65 

.66 

.24 

.64 

R2 

.43 

.13 

.42 

.44 

.06 

.41 

F 

12.60 

4.46 

8.67 

7.12 

6.19 

15.39 

df 

1.17 

1,31 

2,26 

1.9 

1,103 

2,44 

P< 

.003 

.05 

.002 

.03 

.02 

.0001 

Equation 

Var  1 

CompSkl 

MilRes 

MilRes 

ASVAB-ST 

PMOS 

MoColl 

Beta 

.59 

.35 

.83 

.26 

.24 

.57 

t 

3.33 

2.11 

3.10 

2.67 

2.49 

4.88 

P< 

.003 

.05 

.001 

.03 

.02 

.0001 

Var  2 

!  — 

ASVAB-C1 

_ 

MilRes 

Beta 

.16 

.25 

t 

2.83 

2.18 

P< 

.01 

.04 

Table  BS  J 

Regression  of  Objective  Gain  Scores  on  Student  Background  Characteristics 


-  't  .  '  ,  ^  ^  ^ 

*  m,  }  .fei 

Imi 

jgggj 

Step  1 

CompSkl 

TVCrs 

ASVAB-SC 

Reas 

HWEnvir 

No  Data 

Step  2 

ETS 

PMOS 

DutPos 

TVCrs 

Crslnt 

Step  3 

— 

— 

Corres 

ReadHrs 

— 

Step  4 

— 

— 

ETS 

-- 

— 

R 

.73 

.58 

.82 

.72 

.29 

__ 

R2 

.53 

.34 

.67 

.52 

.08 

F 

10.46 

6.85 

10.83 

7.27 

4.71 

df 

2,16 

2,27 

4,22 

3,20 

2,104 

P< 

.003 

.004 

.0001 

.002 

.01 

Equation 

Var  1 

CompSkl 

TVCrs 

ASVAB-SC 

Reas 

HWEnvir 

-- 

Beta 

.62 

-.40 

.61 

-.43 

-.21 

t 

3.62 

-2.57 

4.72 

-2.74 

-2.25 

P< 

.003 

.02 

.0001 

.02 

.03 

Var  2 

ETS 

PMOS 

DutPos 

TVCrs 

Crslnt 

Beta 

-.38 

.39 

-.54 

.43 

-.19 

t 

-2.24 

2.51 

-4.07  j 

2.77 

-1.99 

P< 

.04 

.02 

.0005 

.02 

.05 

Var  3 

— 

— 

Corres 

ReadHrs 

1  Bcta 

-.40 

-.35 

t 

-3.04 

-2.23 

P< 

.007 

.04 

Var  4 

.. 

ETS 

•• 

__ 

Beta 

.31 

t 

2.45 

P< 

.03 
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Table  B5.4 

Regression  of  Subjective  Post  Course  Performance  on  Student  Background  Characteristics 


SSI 

IKS! 

Q| 

Grade 

MoColl 

ASVAB-OF 

No  Data 

No  Data 

cl 

TVCrs 

— 

DutPos 

R 

.54 

.33 

.85 

_ 

_ 

R1 

.29 

.11 

.71 

F 

5.93 

4.30 

8.79 

df 

1,30 

1,36 

2,7 

P< 

.007 

.05 

.012 

Equation 

Var  1 

Grade 

MoColl 

ASVAB-OF 

— 

— 

Beta 

-.44 

-.33 

.79 

t 

-2.80 

-2.07 

3.89 

P< 

.009 

.05 

.006 

Var  2 

TVCrs 

DutPos 

1 

" 

Beta 

.32 

.45 

t 

2.03 

3.85 

.05 

.03 

_ 
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Table  B5.5 

Regression  of  Subjective  Pre-Post  Gain  on  Student  Background  Characteristics 


-s'  -fS  ff’ 

S4lf 

SSi 

III3I 

|||jl 

PH 

ASVAB-OF 

DutPos 

MoPMOS 

None 

Signif. 

No  Data 

— 

PrevExp 

u 

.64 

.85 

_ 

R2 

.41 

.72 

F 

12.64 

21.80 

df 

1,18 

3,26 

{X 

.003 

.0001 

Equation 

Var  1 

ASVAB-OF 

DutPos 

— 

— 

Beta 

.64 

-.57 

t 

3.56 

-5.08 

P< 

.003 

.0001 

Var  2 

MoPMOS 

_ 

.. 

Beta 

-.42 

t 

-4.01 

P< 

.0005 

Var  3 

— 

PrevExp 

— 

— 

1  Beta 

-.25 

3  * 

-2.23 

1  P< 

.04 

mm 

193 


Table  B5.6 

Rating  of  Course  Materials  and  Processess 


Date  of  birth 


Grade 


PMOS  (time  in) 


Intent  to  reenlist 


Interest 


Dist.  Ed/Tech. 


Table  B5.7 

Regression  of  Composite  Rating  of  Course  Materials  and  Processes 
on  Student  Background  Characteristics 


H3 

Cw.  o.  -  .  /  > 

HAZ 

t 

1  Step  1 

None 

None 

MoPMOS 

Crslnt 

None 

Signif. 

Signif. 

— 

— 

Signif. 

R 

.70 

.23 

_ 

R2 

.49 

.06 

F 

7.73 

5.89 

df 

1,12 

1,101 

P< 

.02 

.02 

Equation 

Var  1 

_ 

MoPMOS 

Crslnt 

— 

Beta 

-.22 

.23 

t 

-2.78 

2.43 

P< 

.02 

.02 
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ASVAB 

MM 

ST 


Time  in  college 


College  degree 


Dist.  Ed /Tech, 
prior  TV  course 
prior 

correspondence 


Table  B5.9 

Regression  of  Overall  Course  Ratings  on  Student  Background  Characteristics 


Step  1 

MoColl 

ASVAB-GM 

Credit 

-  ■ 

MoColl 

DOB 

Step  2 

— 

MoPMOS 

MoPMOS 

— 

— 

R 

.48 

.72 

.71 

.20 

.34 

R2 

.24 

.52 

.50 

.04 

.11 

F 

5.22 

11.22 

8.65 

4.54 

5.48 

df 

i,17 

2,21 

2,17 

1,105 

1,43 

P< 

.04 

.001 

.003 

.04 

.03 

Equation 

Var  1 

MoCoil 

ASVAB-GM 

Credit 

MoColl 

DOB 

Beta 

-.48 

-.69 

-.54 

.20 

-.34 

t 

-2.29 

-4.16 

-3.13 

2.17 

-2.34 

P< 

.04 

.001 

.006 

.03 

.03 

Var  2 

Beta 

t 

P< 

MoPMOS 

-.62 

-3.74 

.002 

MoPMOS 

-.43 

-2.53 

.03 
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Appendix  H 

Standard  TNET  Requirements 
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Appendix  H 

Standard  TNET  Requirements 


Standard  TNET  Classroom  Equipment 
Standard  TNET  equipment  provided  for  each  site  consisted  of: 

•  An  antenna  and  feedhom  (2.4  meter  dish  antenna) 

•  A  PC  dedicated  to  communications 

•  A  unit  for  encoding  and  decoding  the  compressed  digitized  signal 

•  Audio  equipment,  including  microphones 

•  A  remotely  controlled  camera 

•  Two  (2)  35”  monitors 

•  A  graphics  stand  with  camera 

•  A  desk-top  controller  unit  to  handle  transmission  and  reception 

•  Cabinet  for  equipment:  74"H,  72"W,  31"D 

•  A  multimedia  computer  system 

Room  Size  and  Configuration 

The  teletraining  room  should  be  a  standard  classroom  approximately  20’  by  25’. 

The  room  should  be  configured  so  that  viewers  sit  no  closer  than  6  feet  from  the 
screen,  with  a  maximum  of  8  times  the  width  of  the  screen  at  an  angle  no  greater  than  45 
degrees. 

The  classroom  must  provide  a  route  for  the  antenna  cable  for  connection  to  the 
VTT  equipment. 

The  room  must  be  able  to  accommodate  the  equipment  in  its  cabinet:  74”H, 

72" W,  31"D,  weight  of  860  lbs.  The  doorway  must  be  wide  enough  to  admit  the  cabinet. 

The  furniture  must  include  a  table  or  tables  for  the  audiographics  system  (pad  and 
computer),  the  graphics  stand,  and  any  other  equipment  used  in  a  particular  application. 

Furniture  should  be  laid  out  to  accommodate  installation  of  microphones, 
microphone  cables  and  antenna  cable. 

Room  Conditions 

When  classes  are  not  in  session,  the  room  temperature  may  range  from  50  to  100 
degrees  Fahrenheit  (10  to  40  degrees  Centigrade).  The  humidity  may  range  from  20  to  90 
percent. 

When  classes  are  in  session,  the  recommended  room  temperature  may  range  from 
65  to  68  degrees  Fahrenheit  in  winter  (18.3  to  20  degrees  C)  and  from  75  to  78  degrees 
Fahrenheit  in  summer  (23.9  to  25.6  degrees  C).  The  relative  humidity  should  be  50 
percent  or  the  design  temperature  equal  to  the  outside  air  dew  point  temperature, 
whichever  is  less. 

The  lighting  should  be  sufficient  for  presentation  of  normal  resident  courses  (30 
foot  candles  for  area  lighting,  50  foot  candles  for  work  station  lighting) _ 
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